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ABSTRACT 


One of the most important missions all Navies have is to constantly and 
sufficiently monitor their area of responsibility. This task becomes more 
challenging when a surveillance system operates in a complex environment with 
high traffic of merchant and fishing vessels and the existence of many islands. 
Potential tactics that targets might use increase the difficulty of this task. 
Integrating Unmanned Aerial Vehicles (UAVs) into a surveillance system that 
consists of ground radars and surface ships might enhance the system’s 
capabilities and mitigate its vulnerabilities. In this study, the extremely complex 
maritime environment of the Aegean Sea is modeled in the Map Aware Non 
Uniform Automata (MANA) agent-based simulation environment to explore the 
effectiveness of UAVs in those conditions. The results from almost 100,000 
simulated Intelligence, Surveillance, and Reconnaissance missions are analyzed 
using descriptive statistics, ANOVA, stepwise regression, and partition trees. It 
was found that by integrating one or two UAVs into a traditional surveillance 
system, it becomes more efficient in the detection and persistent surveillance of 
enemies and neutral targets. The most important factors that affect the 
surveillance system’s performance are the detection capabilities of its sensors, 
the communication accuracy, and the enemy’s counter-detection capability. 
Thus, Greece and other countries with similar geographical characteristics 
should deploy UAVs in a maritime surveillance role. 


THIS PAGE INTENTIONALLY LEFT BLANK 


vi 


TABLE OF CONTENTS 


INTRODUCTION seissesevcvccsscacessnveinsecvecinsasstensvennveusinenseseweunvevsteenvernstetsessasaasicneens 1 
A. GENERAL FRAMEWORK wssscncicccsccenssssesctecsnssctentesseseuscniescenducestsesaceeters 1 
1. The Role of Hellenic Navy in Surveillance...........::ccssssssseeeeee 1 
2. Complexity of the Maritime Environment ..............::::::eeeeeee 3 
3. Vulnerabilities of the Current Surveillance System............... 4 
B. PROBLEM STATEMENT wevesssccsscccsisvstececcevsentecsisenssccessvevnecsvaesncsesexsees 6 
C. PURPOSE-OF THIS:S PUD Y so isitecnatansrssacontaatateerabasalpehasatins aatatinaboretanc 7 
D. SCOPE OF TRB al eats aptel oseaptechcesdecshbust phy taacelesphtasticphamsateantaeieatadens 9 
E. RESEARCH QUESTION AND CORRESPONDING HYPOTHESES... 9 
1. Primary Research Question...........:::::ecccecceesseeeseeeeeeeeeeeeeeeeees 9 
2. Exploratory Research Questions ............::cccssssseeeeeseeneeeenes 10 
F. OVERVIEW OF METHODOLOGY.........::ccccccsseeceeeseeeeeeeeeeeeeeeeeeeeeeeeees 10 
G. BENEFITS OF THE: STUDY ei, ccccecstsuctetsnincdetsadeiaetdntentehiateatatccdeanetess 11 
H. THESIS: FLOW ssciencivescocssvtecccscnctesstewctveseccbeacivesteecisactenctscctescteecteuntes 11 
BACK GROUND ii sisccceccvcsavencvssavencuicevenses seveacyaaavsneusatutacviatvencxsatveacwicevsacwecevens 13 
A. AREA. OF INTEREST wisccesssieeccosssnesccestinesenenensisramecnsaeneecventenenereent 13 
B. PROTECTING NATIONAL SECURITY ........::cccsssseecceesseeeeeeeeeeseeeeeeees 15 
C. CONCEPT OF OPERATIONS ........ccccceteeeceeeeeeeeeeeeeseeeeeeseneeeeeeneeeeeees 18 
D. AGENCIES INVOLVED IN CURRENT SURVEILLANCE SYSTEM.. 20 
1. FIGUISING NOVY wiisccsiaceccstciscciniciscniatuintrinivenenictvercnentvencneseveatvintvans 20 
2: Hellenic Coast Guatd ........::::ccccccccessssseseeeeeeeeeeeeeeeseseeeeeeeeeeees 22 
3. MeNENIC All: FOrC@ nish canadian 23 
4. FISTS IC A LINY sa suseascesndesresa iedecacavedenacederedanevedecaredenedareuedaarneusna 23 
5. NAD sacsencivasssecicesassccenctentaccctevissecceradiassectscuscesaciascuasdewivenndzeceds 24 
6. EUropean UNION jiiiiccstisccwsctucccvestvenwantitcaveutiiranvathencweetveaieentavals 26 
E. AIRBORNE PLATFORMS ......0.......::eecceeeeseeeeeeeeeeeeeeeeeeneeeeeeeeeeeeeeeeeeenes 27 
1. Applications for UAVS .vcciiciiiciiccctnctsivesiecinseretvinneivecimanseacdni 28 
2. TYPOS OF AVS siiccinctonaccechenatbauteannswoteanrnaaeeaseucbencbeadtutahawanoawias 30 
a. Broad Area Maritime Surveillance (BAMS)......+++++++++ 30 

b. Vertical Takeoff/ Landing Tactical Unmanned 
AGrial SVSIGINS jcseteeisrtcctstssanssstsesiendtesadesasseetubsderevvantaas 32 

Cc. Small Tactical/Tier I| Unmanned Aerial Systems 
(STA Sy sissacecacersseracocts iostekin sade Gise tease den actrncinesernanentuekes 35 
F. DETECTION PROBLEM AND RELATED STUDIEG...........::css::ee0 37 
1. Detection Problem .............:::::::::::sssssseeseeeeeeeeeeeeeeeeeeeeeeeeeeeeeees 37 
2: elated Stugies ics ensue tn se lansaien sae aan laue tana lee lai aoe 40 
G. TASK ANALY SIS sais scssssccucessiecves se ianuseessusssatecnavsnascaucasseansssaseasaaerones 42 
MODEL DEVELOPMENT esitecsiasssacccecacacciecenaccczectacsiauesocsseuancesieeesacauexanaasievens 47 
A. MODEL-SELECTION uc ctecitestcoistssaticiesnitestecnitatasaphsutasendidanahatenesdteties 47 
B. MAP-AWARE NONUNIFORM AUTOMATA............:::ccccsseseeeeeeeeseeeees 48 
1. Overview Of MANA wisscccicccccccnccccccciccactce cece ctcececectuececectecueuecteess 48 


vii 


Vi. 


2. MANA CharacteristiCs ..........0:.c:sccecceeceecceeceeccesceeceeseneceeceesenees 49 


C. MODEL IMPLEMENTATION ..........::::s:eecceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeenseeeeees 54 
1. Models ini MAN yctutesticrhcashtablasbhathlarhtatblasbhetblarbtetblasbttbnertes 54 

a. “Open Sea” MOel...:ssccsssesacsostecsssnstessonacdssetivsssuacsaacuase 54 

b. “Many Islands” Model.........:11sssssssssesesssesssesesssesssseesens 55 

2. ASSUMPTIONS wie ccieccicecicccicenienci nuit eeeet est cusieunteusieweteusieusteusnewsteenn 57 

3. Measures of Effectiveness (MOE) ...........::ccssssseeeeeeeeeeeeeeeeeees 57 

4. Agents DeSCription...............:cccccccccccccceeceeeeeenceeeeeeeeeeeeeeceeeeeee 58 

a. BUC SHIPS sorssiassiiccisssteasievedisshiasdssstessouaddassiiasieustnsaiaees 59 

b. BIG@ UAV passcsicpittasnenciipavesiaveveseiatenissbasevarebibavenhivesenetans 61 

Cc. Blue Ground Raa .....s1scccsccssnrtessessnnnesseesenrnneesseneees 62 

d. Blue Operational CONtre ....11cccccccccsesseeeeeseceessssseeseeness 63 

e. FRO SHIDS sivscssstsveretivetesasssctinstecasavesagesnuecasssacesssuaasaaacads 63 

f. Mercnant VOSSCIS. eiicisisesisnsdantcercisustassdsusserndeatsersienseenes 64 

g: FISHING VOSSGIS iin icsoisindeinintadidndeladedneuiwseasdadedeiviaeteadate 65 

DESIGN OF EXPERIMENT isssssocssiessenvcesvsessenescanscnevteaverweenrsnestieeenvmnenervens 67 
A. EXPERIMENTAL SETUP AND FACTORS ........2::cccccseseeeeeeeeeeseeeeeeees 67 
1. EXperimental: SOtup j.sihisis cset pisvtinasessiedestiedinwitsuantesdicebiashewain 67 

2. De@SIQN FaCtOT wi ncis ciciindsscsenwsccavectsanssutcaeccierdeendusndeiwdecctaertneaens 68 

B. NEARLY ORTHOGONAL LATIN HYPERQUBE DESIGNG.............. 73 
C. EXPERIMENT EXECUTION .....00....:ccseeceeeeeeeeeeeeeeneeeeeeseseeeeeeeeeeeeeeeeees 76 
RESULTS AND DATA ANALYSIS isiscscaccesssscscecsccsesecstesinetentvesesontcadesessncesess 79 
A. MEASURES OF EFFECTIVENESS AND DATA PREPARATION .... 79 
1. Measures of EffeCtiveneSs...........c.cssssssseeeeeeeeeeeeeeeeseneeeeeeeeees 79 

2. Data Preparatlom sasiiisisasccda ands cvescnindrastudncneaendnandsendacrdnendaasanawans 80 

3. Assumptions for Statistical Analysis Method..............::000 82 

B. RESULTS OF INITIAL RUNS iierccncssnicstesidaintetectactareiin panera reiuanse eeteias 83 
1. “Open. Sea” Mode) weiccscstecscscntsicencedsciersadacccadedessarendacaencravenenans 84 

a. De@SCriptive StatiStiCS......2::ccccccccccsssssseseseeeeesssssesseeseees 84 

b. Hypotheses TeStin ......+::ssseccccccccccssessnneeeeeeessssssnsseneees 86 

2. “Many islands” Model ...........cccssseeseeeeeeeeeeeeeeeeeeeeseeeeeeeeeeeeeneees 88 

a. DeSCriptive StatistiCS......2::ccccccccscsssssseseeeeeessssssesseseees 88 

b. FIV POINCSES TOSUIAG ss sisssssesssesssesssassossasessssssvasssssanasasnns 90 

C. EXPLORATORY ANALYSIS sx ictcacestatasscisetntspitaladciputasepitudassieelin 92 
1. "Open Sea .MOdel i ciicssiersssasdasncdersimvesntanonadedcauicnetventeandconcie 93 

a. Reegression Amall SIS. .....:sscccccscssesesseeeseeeseeeseeeseeeneeeeeeess 93 

b. ParhiliOn TiO wicscseavirsensavecsersactaseeseeeseerseceerseersnseseertecsd 101 

2. “Many Islands” Model .........:::cccccceseeeseeeeeeeeeeeeeeeeeeeeeeeeeeeeeees 103 

a. Regression Analy SIS......ccccccccceeeeseeeeeeeeeeeeeeeeeeeeeeenennees 104 

b. PartiiOn TOG cc iissestisistisstinstisstenastiadanstiecidsetiociciszeandens 109 
CONCLUSION sisscecescusssoecsessxepeaextstatacensunsseasesuu san bacsecenssanscataseateoturaabaenesuasiess 113 
A. DISCUSSION AND OPERATIONAL INSIGHTS ..........:::ccsssseeeteees 113 
1. Answers to the Primary Research Question .............:::::05 113 

2. Answers to the Exploratory Research Question............... 115 


viii 


B. FUTURE RESEARCA scvisscits cevcssssesienniscstesnvernteusnecnseuniesesarewerrienuensins 116 


C. TAKEAWAY icisccescncisectivieiecinati tess rintinceinestesedvanteneadardestseaedadareuceessiaes 117 
APPENDIX A. COMPARISON BETWEEN NORTHERN AND SOUTHERN UAV 

FOR BOTH OPEN SEA AND MANY ISLANDS MODELG..............:ccccseee 119 
APPENDIX B. DIAGNOSTICS FOR PARAMETRIC ASSUMPTIONS OF MOE 2 

IN OPEN SEA MODEL tisvicsiitscsecsnaucteassicedecctacteencnectcesatederwemansivansaucduereactiens 123 
APPENDIX C. DIAGNOSTICS FOR PARAMETRIC ASSUMPTIONS OF BOTH 

MOES IN MANY ISLANDS MODEL. .....00....cccsceseeeeeeseeeeeeeeseeeeeeeeeseeneeeeenneenes 125 
LIST OF REFERENCES iviscisscsscesccvisessrentssvecnsentennsecstennvernensiennaenwenesaveseuniennensies 127 
INITIAL DISTRIBUTION LISD vriscitsnsaccsininson seis cstesnniedsnwadsutherasensadeainainecsitaiaewaniaionnaa 135 


THIS PAGE INTENTIONALLY LEFT BLANK 


Figure 1. 
Figure 2. 


Figure 3. 
Figure 4. 
Figure 5. 


Figure 6. 
Figure 7. 


Figure 8. 
Figure 9. 


Figure 10. 


Figure 11. 
Figure 12. 


Figure 13. 


Figure 14. 


Figure 15. 


Figure 16. 
Figure 17. 


Figure 18. 


Figure 19. 
Figure 20. 
Figure 21. 


LIST OF FIGURES 


Strategic importance of Greece’s location (From Wikimedia.org)......... 1 
U.S. DoD annual profile for unmanned aerial systems (From U.S. 
Library Ol GOnGress; 2012). xiiakssciscsts pes etinasdvoces octvesinenarecisacheenitectveninnsss 8 
Major shipping routes near Greece (From Safemed Project, 
WWW.SafeMedGIS.OFg)......c-ssscescceceeeteceseeesseeeseecenesseeseneeseeseneseneeeneeeseesess 14 
2012 Map of irregular migration routes in Mediterranean Sea (From 
Interactive map on migration, http:/Awww.imap-migration.org)............ 16 
Detections of illegal border-crossing between BCPs by main 
irregular migration routes (From FRONTEX, 2012). ........ccceeeeeeeeeeteeees 17 


HS AITTITOS (P- 268) (From Hellenic Navy, www.hellenicnavy.gr). . 20 
Hellenic Coast Guard’s typical patrol boat (Panther 57) (From 


MotoMarine shipyards, www.motomarine.Gf). ..........::::eeeeeeeeeeeeeeeeeeeees 23 
Radar BOR-A 550 (From Hellenic Army, Www.army.9f).........:::::eeeee 24 
Diagram of MSSIS (From European Commission, 2008). ..............0 26 
Graphical representation of BAMS Operations (From Northrop 

Grumman, www.northropgruMmMan.COM). .........ccccccceeeeeeeeeeeeeeeeeeeeeeeees 31 
MQ-8B Fire Scout (From America’s Navy, www.navy.mil). ..........2 33 
Images taken by MQ-8B Fire Scout's EO/IR sensors (From 

Northrop Grumman, www.NorthropGrumman.com). .........::::e:0sseeeeeees 34 


The Integrator RQ-21A is recovered with Insitu's SkyHook capture 
rope after its first operational take offs and landings at sea from the 
San Antonio class dock landing ship USS Mesa Verde (LPD-19) in 
the Gulf of Mexico on 10 February 2013 (From Naval Drones, 


WWW NaVvalgrOnleS:COIt): 1h. 02/n0 ele Re rll el esl le bel 36 
Specific attenuation for rain calculated for two model drop size 
distributions (Fromi-Crame,: 1981). oc cecccenccedceenetevecenct chet entreneceneceneteneeat 39 


Range Resolution. The left pair of targets in closer distance is 
detected, but the right one at a greater distance is not (From Radar 
Tutorial, www.radartutorial.Cu). ...........cccccceceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeaes 40 
Ways to study a system (From Law, 2007). ..........ceceeeeeeeeeeeeeeeeeeeeeeeeee 47 
Background Map (left) used in a scenario in MANA. Terrain Map 
(right) for the same scenario. The values that have been set for the 
colors in the Terrain Map are noted in Table 9. ................:csseseeeeeeeeeees 51 
Snapshot of SA Map (left) and Inorganic SA Map (right), concerning 
Squad 1, for the same time of the simulation run. Fellow squad 
members are represented by triangles, other friendly targets are 
represented by inverted triangles, enemies are represented by (red) 
squares and neutral targets are represented by (blue) inverted 
squares. Unknown contacts are represented by white rectangles...... 53 


“Open Sea” model implemented in MANA. .............:::::cecceeeeeeeeeeeeeeeeees 55 
“Many Islands” model implemented in MANA.............:::::c:::eeeeeeeeeeeeees 56 
Personalities tab of the Blue ship agent for the default state.............. 59 


xi 


Figure 22. 
Figure 23. 


Figure 24. 
Figure 25. 
Figure 26. 
Figure 27. 
Figure 28. 
Figure 29. 
Figure 30. 
Figure 31. 
Figure 32. 
Figure 33. 
Figure 34. 
Figure 35. 
Figure 36. 
Figure 37. 


Figure 38. 


Figure 39. 
Figure 40. 


Figure 41. 
Figure 42. 
Figure 43. 
Figure 44. 
Figure 45. 
Figure 46. 


Figure 47. 
Figure 48. 


Sensors tab of the Blue Ship AGent. ........ cc ceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeees 60 
Map Tab of the Blue UAV agent in the “Many Island” model with 

ONG: UAV availa DIG weseicanessncts® actus andoctnont hotavocakioa nt esauatauedaviosssptoseeauadasnences 61 
Inter squad SA table of the Blue Operational Centre...................:005 63 
Sensors tab of the Red ship agent. ............:cccceecceeeeeeeeeeeeeeeeeeeeeeeeeeeeees 64 
Snapshot of the spreadsheet used for the DOE. ..................ccccseeeeeeees 14 
Scatterplot Matrix for all the factors in the DOE...................:eeeeeees 75 


Snapshot of correlation matrix for the first ten factors of the DOE...... 76 
Summary statistics for when a single UAV is launched from 


Northwest or Southeast in the “Open Sea” model. ............::::::ccceeeeeees 81 
Side-by-side boxplots for the four scenarios in the “Open Sea” 
WMO DS ae cadl ead Seved ica cntteadutaas nda tcatns Ataiht quai ana leabd toate taal cavatab etait aeaetas 82 
Histogram of the residuals across the four cases in the “Open Sea” 
WIGS ly scicecvctcecatvottielstas, Coat rttcelaepacuncchemenae Moitea teacteantvoitustavelaeatwataaadve! 83 
Distributions of MOE 1 for the 500 initial runs across the four cases 
in the “Open Sea” MOE. ............ccccccccceeeesseeeeeeeeeeeeeeeesseeeeeeeeeeeeeeeenseees 84 
Distributions of MOE 2 for the 500 initial runs across the four cases 
ITNE: “OPEN SCA” MOGE 5 chxiccscdés weinehovwndnenie vahsevicawire ube Gia einlonbéavkencedeuas 85 


Average percentage of time the targets are detected across the 
four cases of the “Open Sea” model. (Standard Error ranges from 


0.0013 to 0.0027 and would not be easily seen on the bars). ............ 86 
ANOVA for the initial runs of the “Open Sea” model...............:::::::000 87 
Distributions for MOE 1 for the 500 initial runs across the four cases 
in the “Many Islands” Model. .........22---::cccecceneceeecceeeeeeeeeeeteeeeneeeeeeeneeneee 89 
Distributions for MOE 2 for the 500 initial runs across the four cases 
in the." Many Islands” MOGs s:cccccesccestcececeeececeedeeceneCececnnceeseceneeeneceeecest 89 


Average percentage of time the targets are detected across the 
four cases of the “Many Islands” model. (Standard Error ranges 
from 0.0012 to 0.0035 and would be too small to be easily seen on 


LNG: Dts ) esas ates aretcstete tinette ene cian dts liek in duns nasi tie Anite ates 90 
ANOVA for the initial runs of the “Many Islands” model..................00 91 
Model diagnostics for the main effects model prior the Box-Cox 
transformation (“Open Sea”).......2.2.2:s:s:c.sccesecesceeceeeceeeet cece 94 
Box-Cox graph that suggests transformation with A=0.5 (“Open 
A Vis See dae le ee id ee ad Rd et a hii bee Nhs 95 
Model diagnostics for the main effects model after the Box-Cox 
transformation (“Open Sea”). ....2....ccese cess ceeeeseeceeeeeencncceectesetesetccenene 95 
Model diagnostics for the main effects with interactions model 
CODEN: SOA Ye cietschcacetesk era alec an biceed hi oeslexecdecdern levadecscevtn elven ckateres 98 
Model diagnostics for the second order model (“Open Sea’). .......... 100 
Partition tree for the “Open Sea” model after seven splits ................ 102 
Split history for the Partition tree of the “Open Sea” model. ............. 102 
Model diagnostics for the main effects model (“Many Islands”). ....... 104 
Model diagnostics for the main effects model with interactions 
CrIVlemyISIANGS. cous tee ieustentccetdensdousdonteene cevecces eee leuk cckcecesderbcceb dap Sooebeet 106 


Figure 49. Model diagnostics for second order model (“Many Islands’). ........... 108 
Figure 50. ‘Partition tree for the “Many Islands” model after six splits. ............... 110 
Figure 51. = Split history for the Partition tree of the “Many Islands” model. ........ 110 


xiii 


THIS PAGE INTENTIONALLY LEFT BLANK 


Xiv 


Table 1. 
Table 2. 
Table 3. 
Table 4. 
Table 5. 
Table 6. 
Table 7. 


Table 8. 
Table 9. 


Table 10. 
Table 11. 
Table 12. 
Table 13. 
Table 14. 
Table 15. 
Table 16. 
Table 17. 
Table 18. 
Table 19. 


Table 20. 
Table 21. 


Table 22. 
Table 23. 
Table 24. 
Table 25. 
Table 26. 


Table 27. 


LIST OF TABLES 


Information about maritime transport (From European Commission 


TUNG 2 OTS Yat ta st oer celica ead tecad ded code ax dy oust eede eeceet gehen teed dened 15. 
General characteristics of HS MACHITIS (Type Osprey HSY 56A). .. 21 
General characteristics of Panther 57 ................ccccseeeeeeeeeeeeeeeeeeeeeeeeees 23 
Combatant commander/service UAS mission prioritization matrix 

(From Office of the Secretary of Defense, 2005)............... eee 29 


Specifications of RQ-4 Global Hawk Maritime Demonstration 
System (From Northrop Grumman, www.northropgrumman.com). .... 32 
Specifications of MQ-8B Fire Scout (From Northrop Grumman, 


www.NorthropGrumman.Com). ........:cceeeeeeeeee eee eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeees 34 
Specifications of Integrator (RQ-21A) (Insitu, 2012 ). oe eeeeeeeeees 37 
Maritime surveillance task analySiS. .............:cccccccceeceeeeeeeeeeeeeeeeeeeeeeeees 43 
Values for Going/Cover/Concealment of the Terrain Map depicted 
WY FIQUIC Tl Puce leeatccbtivad tecsuasaua nuasivanametetars ecm itce late ou areca aiateta alee 51 
Agents used in the “Open Sea” and “Many Islands” models .............. 58 
EXPOCriNIGMl SCP cai vcteseiciivintesciieeciibed eodadies blew alvahies ts nehedededbeownieerers 68 
Independent factors varied in the DOE.................:cccccceeeeeeeeeeeeeeeeeeeeees 69 
Dependent variables in the DOE ..................ccccccccceeeeeeeeeeeeeeeeeeeeeeeeeeees 72 
Ordered pairwise differences report after Student’s ttest (“Open 
c=) ee a epee ote 88 
Ordered pairwise differences report after Student’s ttest (“Many 
FE gt oi) ieee test eee eeun sic neeeerinnetee Mra tion runtcee Went iuptice euneine Meas inne coerce 92 
Statistical results for the main effects model after the Box-Cox 
Lransionmalion: (COPS Sea lice ea a ae ladle 96 
Sorted parameter estimates for the main effects model after the 
Box-Cox transformation (“Open Sea”). ..........ceeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeees 97 
Statistical results for the main effects model with interactions 
(OBR: SCA) Gees cece rete site de nathan teat ee usr tases ec tetas ute tute tute Sate e ed out tute 98 
Sorted parameter estimates for the main effects model with 
interactions (“OPEN SOa")......ccccccccccceneceneceneceseceneeeneteneceneceneeseecenseneeeenee 99 
Statistical results for the second order model (“Open Sea’). ............ 100 
Sorted parameter estimates for the second order model (“Open 
TOE Vico ite once ale ted ae ned arch Ga Ben seth rete Sih rate Sel aan leks Al ord oxo 2 hehe an 101 
Variable contribution for the Partition tree of the “Open Sea” model. 103 
Statistical results for the main effects model (“Many Islands’).......... 105 
Sorted parameter estimates for the main effects model (“Many 
ISIE Sess Seow Se ta ee eh ee 105 
Statistical results for the main effects model with interactions 
CiMlanys|SlanGs” \sic coat t eeadercdedececedeuedevedevece vedere eebceeecdbcceveteumeperceretes 107 
Sorted parameter estimates for the main effects model with 
interactions. (“Many Islands”). .......2-.:---cc-cccceeeeeteceeeeeeeneseeeeeneeeneteneeetes 107 
Statistical results for the second order model (“Many Islands’). ....... 108 


XV 


Table 28. 


Table 29. 


Sorted parameter estimates for the second order model (“Many 


ISAS yatta etsncte. bdseidere ta tes. Gasset Ade adel Au sate tcated ets thie ei ade hie Reds 109 
Variable contribution for the Partition tree of the “Many Islands” 
PTVOG Sl poiasavties cant casts Guaasas wentacnacaldrauaaeuersananstaasaasalashsamaaaaauns danas eaiamaaeiies 111 


xvi 


Als 

AOl 
AWACS 
BAMS 
BDA 
COP 
DoD 
DOE 
DTS 
EO/IR 
EU 

FOV 
FRONTEX 


GCS 
GHMD 
HN 


ISR 

LH 
MANA 
MCCIS 
MDA 
MOE 
MPA 
NATO 
NOLH 
RADAR 


LIST OF ACRONYMS AND ABBREVIATIONS 


Automatic Identification System 
Area of Interest 

Airborne Warning and Control System 
Broad Area Maritime Surveillance 
Battle Damage Assessment 
Common Operational Picture 
Department of Defense 

Design of Experiment 

Discrete Time Simulation 
Electro-Optical/Infrared 

European Union 

Field of View 


from French: Frontiéres extérieures (legally: European 
Agency for the Management of Operational Cooperation at 
the External Borders of the Member States of the European 
Union) 


Ground Control System 

Global Hawk Maritime Demonstration 

Hellenic Navy 

Infrared 

Intelligence, Surveillance, and Reconnaissance 
Latin Hypercubes 

Map-Aware Non-uniform Automata 

Maritime Command and Control Information System 
Maritime Domain Awareness 

Measure of Effectiveness 

Maritime Patrol Aircraft 

North Atlantic Treaty Organization 

Nearly Orthogonal Latin Hypercubes 

Radio Detection and Ranging 


xvii 


RMP 
SA 
SAR 
SEAD 
SEED 
SSC 
STUAS 
ToT 
UAS 
UAV 
VTUAS 
VTUAV 


Recognized Maritime Picture 

Situational Awareness 

Search and Rescue 

Suppression of Enemy Air Defense 
Simulation Experiments and Efficient Design 
Surface Search and Control 

Small Tactical Unmanned Aerial System 
Time on Task 

Unmanned Aerial System 

Unmanned Aerial Vehicle 

Vertical Takeoff Unmanned Aerial System 
Vertical Takeoff Unmanned Aerial Vehicles 


xviii 


THESIS DISCLAIMER 


The reader is cautioned that the computer programs presented in this research 
may not have been exercised for all cases of interest. While every effort has 
been made within the time available to ensure that the programs are free of 
computational and logical errors, they cannot be considered validated. Any 
application of these programs without additional verification is at the risk of the 
user. 
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EXECUTIVE SUMMARY 


One of the most fundamental missions of the Hellenic Navy is surveillance of its 
territorial waters, as well as the waters in its region. This mission is undoubtedly 
of strategic importance taking into consideration the geographical region where 
Greece is located, at the intersection of three continents (Europe, Asia and 
Africa), as well as Greece’s participation in the European Union (EU) and North 
Atlantic Treaty Organization (NATO). 


Monitoring the Aegean Sea, the Eastern area of the Mediterranean Sea, is 
by default difficult because it demands cooperation among different sensors. 
These sensors might be ground radars, patrol ships, air force reports, and 
various electromagnetic or electronic detections. The complexity of the maritime 
environment increases the difficulty in the surveillance. The most distinctive 
geographical characteristic of the Aegean Sea is the number of islands located 
there. Despite its relatively narrow area, the Aegean Sea accommodates more 
than 3,000 islands, islets, and rocks. Furthermore, this sea is used by 
approximately 55,000 transiting merchant vessels on an annual basis. 


The current maritime surveillance system in Greece, despite its cost and 
its demands on human resources and material, is highly vulnerable due to a 
number of factors. First of all, the weather conditions affect the current sensors in 
various ways. For example, high sea states may prevent small surface ships 
from patrolling. Moreover, low visibility does not allow the visual observers to 
locate targets at longer distances. Similarly, in rainy conditions radars are 
negatively affected, resulting in smaller detection ranges. Another limiting factor 
is location. “Blind sectors,” the areas that are not covered by the ground radar 
systems, result from the way systems must be positioned. Finally, the 
adversary’s tactics may diminish the detection range. Such tactics include two 


targets sailing very close to one another or vessels staying close to the shoreline. 
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Greece is facing a major challenge to cope with the large influx of mixed 
migratory flows (including irregular migrants, refugees, and asylum seekers) and 
the current economic crisis. According to EU, Greece has become the main entry 
point for irregular migratory flows into the European Union. In 2010, more than 
132,000 third-country nationals were arrested in Greece, including 53,000 in the 
Greek-Turkish border regions. During the first ten months of 2012, over 70,000 
arrests occurred, including about 32,000 at the border with Turkey. People came 


from 110 different countries, the majority from Asia and the Middle-East. 


Conventional wisdom states that Unmanned Aerial Vehicles (UAVs) offer 
two main advantages over manned aircraft: they are considered more cost- 
effective, and they minimize the risk to a pilot’s life. This study investigates the 
ways in which UAVs can be integrated into a traditional surveillance system that 
utilizes only ground radars and surface ships. 


Using an agent-based simulation environment called Map-Aware Non- 
uniform Automata (MANA), two representative areas of the Aegean Sea are 
modeled for our research; the “Open Sea” model, where a ship has to sail for 
about 50 nautical miles without the “coverage” of nearby islands, and the “Many 
Islands” model, where several islands are very close to one another. The first 
represents a typical area of the Northern and Southern Aegean Sea and the 
second of the Central and Southeast Aegean Sea. These rare geographical 


conditions compose a challenging environment operationally. 


In our simulation the Red Force consists of four surface vessels. These 
vessels attempt to transport illegal immigrants into Blue’s area of responsibility. 
To increase the possibility of going undetected, they keep close to nearby 
merchant vessels that have approximately the same course with them (“Open 
Sea” model), or they try to take advantage of islands by sailing close to the 
shoreline during their route (“Many Islands” model). 


Following the base model, a Nearly Orthogonal Latin Hypercube (NOLH) 


is created to efficiently explore the effect of 21 variables on the percentage of 
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time the targets are positively identified. These 21 factors were chosen based on 
opinions of subject matter experts, as well as the author’s experience. In addition 
to the 21 independent variables that are explicitly varied in the Design of 
Experiment (DOE), there are an additional 52 variables that depend on one of 
the aforementioned factors. These variables describe the sensors’ capabilities. 


The simulation runs show that integrating one or two UAVs into a 
traditional surveillance system makes it more efficient in the detection and 
persistent surveillance of enemies and neutral targets. This conclusion applies to 
all the areas of the Aegean Sea, both the Northern and Southern Aegean Sea 
(modeled by the “Open Sea” model) and the Central and Southeast (modeled by 
the “Many Islands” model). 


In all the cases we studied, we consistently found that a traditional system 
with only surface ships and ground radars performs worse than a system that 
uses UAVs. Additionally, we explored a “futuristic” scenario in which only two 
UAVs support the ground radars, without the use of surface ships (no Blue ships 
scenario). The results show that we can obtain similar results with the case 
where we deploy one UAV and two surface ships in support of the ground radars 
(1-UAV scenario). This conclusion indicates that decision makers should review 
the surveillance policy. They can shift from the cost-ineffective use of surface 
ships towards the use of UAVs to better monitor maritime areas. This solution 
might be more beneficial in adverse weather conditions or when there is a lack of 


operational surface units. 


This study also demonstrates that the potential tactics the enemy might 
use to avoid detection cannot mislead UAVs. Whether the enemies try to take 
advantage of nearby merchant vessels and sail close to them to transit long 
distances, or to use the shorelines of numerous islands as coverage, a 
surveillance system that deploys UAVs is able to present significantly better 
performance than a system that uses only ground radars and surface ships. 
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Furthermore, this thesis shows that the area from which a UAV is 
launched affects the performance of the whole surveillance system. As it is easily 
understood, the more contacts that exist close to the launch site, the more the 
UAV contributes to the surveillance system. That is, if the launch site is close to 
areas with high traffic, the UAV will have more opportunities to locate the targets. 
Additionally, the targets will be identified earlier. Therefore, it is extremely 
important to define the best launch site for the UAVs. From the literature review, 
we found that UAVs can be launched either from ground-based sites or 
appropriately equipped surface ships. Both solutions must be examined for 
strengths and vulnerabilities. 


This study also shows that UAVs can mitigate a vulnerability that ground 
radar systems inherently have: the inability for fixed radars to cover their “blind 
sectors,” which result from the way systems must be positioned. Because 
monitoring the whole range of the Area of Interest (AOI) with ground radars is not 
practical (it would require twice as many sensors), the existing radars are placed 
in areas with the most traffic. As a consequence, these systems are not always 
stationed at the highest point of an island, which results in a limited field of view. 
Additionally, taking into account that radar technology is subject to technical 
restrictions, the detecting capabilities are further narrowed. UAVs can easily 


cover those areas providing information about existing targets. 


Finally, this thesis found the important factors that contribute to the 
maritime surveillance system. The two models that modeled corresponding 
environments in terms of transiting merchant vessels, the existence of nearby 
islands and the enemy’s tactics were studied separately, but most of the input 
variables appear in both of them. 


° The detection range of all the sensors that are part of the 
surveillance system seems to have the most significant positive 
effect on the percentage of time the targets are classified. The 
partition tree for the “Open Sea” model highlights the importance of 
having the radars, the UAV, and the ships’ detection ranges larger 
than 22, 19 and 16 km, respectively. 
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Communication accuracy of the Operational Centre has a 
significant positive effect on the system’s performance. The 
partition tree for the “Many Islands” model suggests a value of at 
least 78 percent to better monitor the area of interest. It worth 
noticing that the Operational Centre is crucial in the communication 
network that we established because its role is to gather the 
information from each one of the sensors and send them to the 
patrolling units. 


Enemy’s stealth is a factor that has a significant negative impact on 
the percentage of time a target is classified. Enemy’s stealth is 
presented in both the scenarios we studied where a target used a 
specific tactic to avoid being detected by the surveillance system. In 
reality, this factor describes the enemy's attempt to avoid the visual, 
infrared (IR), and radar contact using various methods. 


The number of merchant vessels and their speed has a negative 
effect on the surveillance system’s performance only in the case 
where the enemy tries to use them as a “Trojan Horse” to transit 
long distances. In environments where there is high traffic, either 
from merchant or fishing vessels, but the enemy does not try to 
take advantage of it, it seems that the surveillance system is not 
affected. 


Average Times between Detections of our own sensors are 
additional important factors in terms of the surveillance system’s 
detection capability. All of them positively affect the percentage of 
time the targets are classified, regardless of the operational 
environment in which the system operates. 


The enemy's speed is a factor that has a significant negative effect 
on the percentage of time the targets are classified. This factor 
emerged only in the “Open Sea” model where the target has to 
transit long distances being “exposed” to the adversary’s 
surveillance system. This is operationally reasonable because 
when a target sails very close to the shoreline to avoid the 
detection, it cannot maintain a high speed due to navigation 
dangers. 


UAV speed has a positive effect on the system’s performance only 
in the case where it has to cover large areas. This study showed 
that when the UAV had to cover areas with a large number of 
islands, its speed was not important. 
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l. INTRODUCTION 


A. GENERAL FRAMEWORK 
1. The Role of Hellenic Navy in Surveillance 


The geographical region where Greece is located, at the intersection of 
three continents (Europe, Asia and Africa), is undoubtedly of strategic importance 
in surveillance (Figure 1). Traditionally, Greece’s foreign policy focuses on two 
major priorities. The first priority is to employ foreign policy as “an operational 
instrument for solving problems both in the region, where Greece is located, as 


well as in the wider international system.” 





Figure 1. Strategic importance of Greece’s location (From Wikimedia.org). 


The second priority is “to contribute actively to promoting peace, stability 
and cooperative patterns of behavior in the region of Southeastern Europe and 
most specifically in the Balkans, and the area of the Eastern Mediterranean” 
(Kranidiotis, 1998). 


Working in this context, the Hellenic (Greek) Navy's mission includes 
“contribution to the deterrence against any potential aggressor and the unabated 
support to Allied, European Union and other international efforts so as to fulfill 
Greece's commitment as a dedicated contributor to North Atlantic Treaty 
Organization, European Union, and United Nations, maintaining readiness to 
effectively contribute to defense and security of the Nation” (Karamalikis, 2008, 
Dd), 


One of the most fundamental missions of the Hellenic Navy is surveillance 
of the territorial waters, as well as the national waters that are close to them. 
This mission is difficult because it demands cooperation between different 
sensors. These sensors might be ground radars, patrol ships, air force reports, 
and various electromagnetic or electronic detections. A serious parameter for the 
Hellenic Navy is the degree of cooperation with the Coast Guard and/or the 


sharing of information with international stakeholders. 


The importance of this mission is reinforced by Greece’s participation in 
the European Union (EU). In the context of EU enlargement, Europe’s maritime 
borders have now expanded with Europe’s coastline to contain 85 percent of the 
EU’s international borders. From the EU’s perspective, this expansion will require 
increased surveillance to tackle problems such as illegal immigration and other 
illicit activities linked to organized crime, such as drug smuggling, human 
trafficking and the trafficking of illicit materials such as WMD and explosives. 
Effective monitoring of EU external borders requires increased cooperation 
between relevant stakeholders, such as coast guard organizations and law 


enforcement agencies, in maritime surveillance. 


The Mediterranean Basin is a strategically important region for the EU, 
and it is necessary to construct a strong economic area capable of contributing to 
the Union's regional balance by assuring peace, stability and prosperity. The 
Barcelona Initiative was launched with these objectives in mind and is now a 
central element of the EU-Mediterranean Policy. Furthermore, the Hellenic Navy 


aims to be vigilant in detecting all threats that might cause instability in the area 
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in which it operates. Those threats, like terrorism, organized crime, and political 
or economic crises must always be taken into serious consideration at the 
campaign level, as well as at the operational level (Kyriazis, 2004, pp. 64-66, 68, 
71-73). 


Traditionally, the Navy handles the surveillance mission with multiple 
assets under a broad mission called Surface Search and Control (SSC). At sea, 
surface detections are often made by medium-range ground radars scattered on 
numerous islands which provide optimal coverage with respect to sensor 
capabilities and operational priorities. Additionally, a large number of surface 
ships are assigned to patrol comparatively small sectors in order to contribute to 
either the detection or the identification task. 


Besides the Navy assets, ground-based or surface, there are additional 
means from other military branches or organizations that provide useful 
information to the Hellenic Navy which has the coordinating role. A detailed 
presentation of all the stakeholders who participate in the current surveillance 
system is provided in Chapter Il. 


Realistically, not all the assets are available at all times. All the information 
received from these assets is used to form and maintain the Recognized 
Maritime Picture (RMP). The RMP is “about maintaining an unambiguous and 
timely database of the position and identification of all tracks, both warship and 
merchant, and being able to distinguish good or cleared ships from the 
adversary, unchallenged, suspect, or blockade running ships” (Germain, 1997). 
The RMP assists the commanders in obtaining the intelligence needed to use 


their assets more effectively and consequently achieve their mission. 


2. Complexity of the Maritime Environment 


The Aegean Sea, which is the Eastern maritime frontier of European 
Union, is the area in which the Hellenic Navy mostly operates. The most 
distinctive geographical characteristic of Aegean Sea is undoubtedly the number 
of islands located there. Despite its relatively narrow area, the Aegean Sea 
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accommodates more than 3,000 islands, islets and rocks (Acer, 2003). 
Furthermore, this sea is used by transiting merchant vessels sailing towards the 
Black Sea (Northeast), Red Sea (South) and Middle East (East). It is estimated 
that approximately 55,000 merchant vessels cross this area annually (Camci, 
Eldemir, Uysal, & Ustun, 2009, pp. 424-429). If we take into consideration the 
large number of fishing vessels (both Greek and Turkish) and sailing boats, 
especially during the summer, one can conclude how difficult it is to monitor this 
Area of Interest (AOI). This task is further complicated when a “target” uses 


unusual tactics to mislead the assigned surveillance sensors. 


3: Vulnerabilities of the Current Surveillance System 


The current maritime surveillance system in Greece, despite its cost and 
its demands on human resources and material, is highly vulnerable due to a 
number of factors. First of all, the weather conditions affect the current sensors in 
various ways. For example, high sea state prevents small surface ships from 
patrolling. Moreover, low visibility does not allow the visual observers to locate 
targets at large distances. Similarly, in rainy conditions radars are negatively 
affected, resulting in smaller detection ranges (Schneider & Williams, 1977, 
pp. 11-29). 


Another limiting factor is location. “Blind sectors,” the areas that are not 
covered by the ground radar systems, result from the way systems must be 
positioned. Since monitoring the whole range of the AOI with ground radars is not 
practical (it would require twice as many sensors), the existing radars are placed 
in areas with the most traffic. As a consequence, these systems are not always 
stationed at the highest point of an island, which results in a limited field of view. 
Additionally, taking into account that radar technology is subject to technical 
restrictions, the detecting capabilities are further narrowed. 


Finally, the adversary’s tactics may diminish the detection range. Such 


tactics include two targets sailing very close to one another or close to the 


shoreline. In such cases the resolution of the radar might not be adequate to 
distinguish the targets (Wehner, 1994; Hutchinson, 2003). 


Furthermore, these limiting factors are multiplied by the complex 
environment in which maritime activities occur. This complexity provides the 
targets involved in illegal activities or adversary warships that are enemies of the 
North Atlantic Treaty Organization (NATO), the EU or Greece, with potential 
ways to avoid detection. 


As mentioned above, an extremely large number of people and operating 
subsystems (units) have to collaborate in the current surveillance system. The 
existing system involves people from different branches (Navy, Army, Air Force) 
with different backgrounds (personnel coming from conscription in contrast to 
voluntary military members), as well as from different countries and security 
organizations. This diversity may be the biggest vulnerability of the surveillance 
system, even bigger than human error or the deficiencies of the existing means. 
Any new system should take into account the leveraging of differences among 
participants to produce innovative, synergistic solutions and balancing of 
divergent stakeholder concerns (Hardy, Lawrence, & Grant, 2005). 


Adding to the challenge, although the cooperation with the international 
organizations (NATO, EU) is exceptional, this cooperation cannot be exclusively 
relied upon, because the majority of the RMP concerning the Aegean Sea 
actually is drawn from the Greek reports. In other words, the majority of the 
information flows from the Hellenic Navy to the other stakeholders rather than the 
other way around. That reinforces Greece’s responsibility to its allies for 
providing near-to-real time and accurate Situational Awareness (SA) in the AOI. 


Furthermore, the participation of the EU’s units in patrolling the area of 
operations definitely assists in the task of surveillance. Even so, the inherent 
problems of that system, such as vulnerabilities due to weather conditions, 
fatigue of the crew, restriction of the detection devices (e.g., radar, night vision 


goggles) remain a challenge to the Greek Navy. Additionally, EU assistance 
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holds only for a period of time throughout the year (two to three months 


annually). 


Finally, the financial crisis in Greece resulted in a significant budget cut in 
the Ministry of Defense. As a consequence, systems that are economical and, at 
the same time, highly effective must be adopted throughout all the sectors, 
including the Hellenic Navy. Although the exact cost for the surveillance of the 
Aegean Sea is unavailable and difficult to estimate (because it involves a 
tremendous number of people and manned devices or units), the cost certainly is 
significant. Therefore, budget cuts to surveillance of the Aegean Sea prompt the 


Greek Navy to derive more cost efficient methods of surveillance. 


B. PROBLEM STATEMENT 


The greatest challenge for the current surveillance system, which does not 
involve Unmanned Aerial Vehicles (UAVs), is to detect comparatively small 
targets (with small Radar Cross-Section). As stated earlier, a common technique 
of small targets is to stay close to the shoreline. Current radar resolution cannot 
distinguish the presence of a small ship so close to the shoreline. This problem 
arises when hostile, relatively small warships (like fast attack boats) or other 
small boats, which might participate in illegal transactions (e.g., human 
trafficking, smuggling, etc.), try to take advantage of this situation and avoid 


being detected by the Greek Naval surveillance system. 


A second problem arises when small hostile warships are assigned to 
transit an “open sea,” which is considered an area with no island for about 50 
nautical miles. In the open sea, small ships are extremely exposed and can 
readily be detected by their adversary’s assets (e.g., air assets, frigates, ground 
radars, etc.). Additionally, small ships are unable to protect themselves because 
they lack anti-air weapons and self-defense systems or missiles. Thus, these 
small warships commonly take advantage of the presence of large merchant 
vessels/tankers and remain as close to them as they can to avoid detection. The 
nature of the Aegean Sea favors this practice. 
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C. PURPOSE OF THIS STUDY 


The purpose of this study is not to build a completely new surveillance 
system, but to introduce the potential benefits of using new technologies/sensors 
which complement the current system. For this reason, we will try to explore the 
ways in which UAVs can be engaged in this process. This application will be an 
innovation for the countries of the East Mediterranean that currently do not use 


UAVs in surveillance of maritime areas. 


The primary reason that we look into this solution is because UAVs are 
increasingly utilized by developed countries in the global operational environment 
today (Frederick, 2006), and this trend is expected continue in the years ahead. 
Additionally, the use of UAVs is believed to mitigate the inherent vulnerabilities of 
the current surveillance system mentioned earlier, as well as reduce the overall 
cost of surveillance. Taking the U.S. Navy as an example, it intends to increase 
the number of UAVs in service while at the same time reduce the number of 
operators (Liu, Wasson, & Vincenzi, 2009, pp. 795-810). Another example is the 
replacement of helicopters by Vertical Takeoff Unmanned Aerial Vehicles 
(VTUAVs) in the U.S. Navy's planned 56-ship fleet of Littoral Combat Ships 
(Burgess, 2004, pp. 24—25). 


Undoubtedly, the global investment in unmanned systems accelerated 
following 9/11 and UAV successes in Iraq and Afghanistan. During the decade of 
2005-2015, the global market for UAVs (including all air vehicles, ground control 
equipment and payloads) is an estimated $16 billion. The U.S. Department of 
Defense (DoD) annual profile for Unmanned Aerial Systems (UAS) is depicted in 
Figure 2. Europe could spend over 1 billion euros on procurement and possibly 


more on research and development (Dickerson, 2007, pp. 114-116). 
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Figure 2. U.S. DoD annual profile for unmanned aerial systems (From U.S. 
Library of Congress, 2012). 


There have been numerous studies that demonstrate the effectiveness of 
UAVs in border security (Yildiz, 2007) and homeland security (Myers, 2007; 
Weiger, 2007). An additional fact that proves the need for aerial means in the 
AOI comes from the NATO contribution to Greece with Airborne Warning and 
Control Systems (AWACS) aircraft during the 2004 Olympic Games (Graham, 
2004). At that time, although there was a fleet of eight surface ships patrolling the 
Aegean Sea (in national and international waters) it was considered necessary to 
have in addition one air asset with advanced capabilities. In comparison to that 
time, perhaps the current need for constant coverage of the AOI is not so 
obvious, and the assets assigned are definitely fewer than in the past. For all 
these reasons we will try to answer similar questions from the perspective of 


operating in a complex maritime environment. 


This study investigates whether a surveillance system with the use of 
UAVs performs better than a traditional system. In other words, we explore 
whether UAVs are capable of mitigating all or part of the vulnerabilities of the 


current sensors (e.g., ground radars, patrol ships, etc.). Additionally, we explore 


what factors should be considered in designing more effective surveillance 


systems that meet the needs of the geographical area in which they will operate. 


D. SCOPE OF THESIS 


This thesis focuses on reinforcing the surveillance system in complex 
maritime environments where high maritime traffic is met and where targets may 
take advantage of adjacent islands or transiting merchant vessels to avoid 
detection. Furthermore, it is assumed that the current surveillance system does 
not involve UAV capabilities, but the benefit of UAV capabilities will lead to their 


implementation with current sensors in the future. 


The scope of this study is not to build a new surveillance system from the 
beginning, but to suggest new elements that could easily be incorporated into the 
existing system. These elements will fill in the gaps and mitigate the 
vulnerabilities that have already been recognized in the current model and will 


form a solution that meets the surveillance objectives being posed. 


This study investigates the extent to which the Hellenic Navy will benefit 
from integrating UAVs into the existing surveillance system in its AOI. For this 
reason, the sensors that currently are involved in this task are studied in 
compliance with the complex maritime environment in which they operate, like 


the Aegean Sea. 
E. RESEARCH QUESTION AND CORRESPONDING HYPOTHESES 


1. Primary Research Question 


Does the use of UAVs in a surveillance role mitigate the limitations of the 
current surveillance system to detect targets in complex maritime environments, 
assuming that the target's tactic is to approach transiting merchant vessels or 
take advantage of nearby shorelines? 


Hypotheses for the Primary Research Question: 


e Ho: There is no difference in the surveillance system’s performance 
whether it uses UAVs or not assuming that the target’s tactic is to 
approach transiting merchant vessels or to take advantage of 
nearby shorelines. 


° Ha: A surveillance system with UAVs performs better than one 
without UAVs assuming that the target’s tactic is to approach 
transiting merchant vessels or to take advantage of nearby 
shorelines. 


2. Exploratory Research Questions 


What are the important factors that contribute to maritime surveillance 
when UAVs are used? 
e Is the number of transiting neutral vessels an important factor? 


° Is the length of the shoreline an important factor for the surveillance 
system’s performance? 


F. OVERVIEW OF METHODOLOGY 


To explore the problem posed, we use an agent-based simulation 
environment called Map-Aware Non-uniform Automata (MANA). In this simulation 
tool we set up a model that represents the current surveillance system. This 
model is compared, applying appropriate statistical techniques, with another 
which uses UAVs in addition to the sensors the previous model used. 


The input data are assembled by reviewing technical data, manufacturer 
supplied information, article reviews, and subject matter expert interviews. Since 
the input values are not always fixed or known, a range of them is used. In this 
way, the uncertainties of the inputs are mitigated, and some of the stochastic 
effects of the warfare are captured (Abel, 2009). A Nearly Orthogonal Latin 
Hypercube (NOLH) design is used to create a pattern of different starting 
conditions. For each condition, a simulation is run for a sufficient number of 
replications to estimate the variability in the output. Exploratory data analysis 
techniques are used to characterize the impact of surveillance sensors and 


tactics on the overall performance of the surveillance system. 
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Our simulation model captures the following attributes: 


e Enemy targets try to take advantage of the presence of transiting 
merchant vessels and the nearby shoreline at all times. 


e All the other sensors, besides the UAVs, have the same capabilities 
either with or without the presence of UAV. 


° The assigned UAV covers those areas that the ground radars are 
incapable of covering. 


The assumptions of the models we create are presented in detail in 
Chapter III. 


G. BENEFITS OF THE STUDY 


This study will benefit the decision makers, either officers or politicians, 
who need to understand how UAVs might be force multipliers. In other words, our 
research will test the value of UAVs in monitoring complex maritime 
environments and provide the initial guidelines for the use of UAVs in the 
maritime domain. Additionally, results from this study will assist decision makers 
in forming the future policy and/or tactics, techniques and procedures for small 
friendly warships to counteract UAVs. 


We are expecting that the results of this study will be applied in similar 
surveillance situations in different domains. A potential area of exploration could 
be the monitoring of maritime traffic in the vicinity of ports. Another area could be 
in border security when combining ground, aerial and maritime sensors. This 
solution could be investigated for situations where potential illegal activities might 
happen using various techniques to mislead the detection sensors both on the 


ground and in the sea or rivers. 


H. THESIS FLOW 


Chapter Il offers a description of the AOI and of the concept of 
surveillance operations. It continues with a discussion of the illegal activities that 


may be encountered. Information about the agencies and units involved in the 
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current surveillance system in Greece is also provided. Finally, the airborne 
platforms currently in use by the U.S. Navy that might be fitted in the case 
studied are described. 


Chapter III begins with an overview of the modeling tool, MANA, and the 
scenario description. Moreover, it provides a detailed description of the model 
and the way it is built. 


Chapter IV presents the design of experiment, as well as a description of 


the variables used in the analysis. 


Chapter V starts with an overview of the analytical techniques being used. 
It continues with the data obtained from the simulation, and it closes with the 


analysis section using appropriate statistical techniques. 


Chapter VI concludes this thesis, summarizing the results, giving a 
thorough discussion of the analysis and offering follow-on problems and 
recommendations. The complete analytical work is contained in the Appendices. 
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ll. BACKGROUND 


A. AREA OF INTEREST 


Greece is an EU Member State in South-Eastern Europe and is mostly 
surrounded by sea. It has a coastline on the lonian Sea in the west, the Adriatic 
Sea in the northwest, the Aegean Sea in the east and the Eastern Mediterranean 
in the south and southeast. Its coastline length is an estimated 17,400 km. In 
Greece’s territory there exist about 10,000 islands and islets making up around 
70 percent of country’s coastline. Greece has a 6 nm territorial sea, but has 
repeatedly declared that “it reserves its legitimate right under international law to 
establish a 12 nm territorial sea at a time deemed appropriate” (European 
Commission Study, 2013). 


Greece’s most significant maritime activities are maritime transport, 
marine and coastal tourism, and fisheries. Greece has the largest merchant fleet 
in the EU and one of the largest merchant fleets in the world. The country is 
surrounded by a rather large number of important shipping lanes (Figure 3) and 
has 20 ports that process more than one million tons of cargo per year (European 
Commission Study, 2013). 
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main ports where crude oil tankers have been 
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main ports activity for year 2005 


Main traffic routes for year 2005 


Figure 3. Major shipping routes near Greece (From Safemed Project, 
www.safemedgis.org). 


The traffic in the Greek seas is enhanced by commercial shipping among 
the ports, recreational boating, as well as ferry transportation. Additionally, 
Greece is a popular tourist destination (ranked 15th worldwide). Tourism is 
extremely important for the Greek economy because it contributes more than 18 
percent to the annual Gross National Product. A large number of cruise ships sail 
the Greek waters, especially during summer. 


Given Greece’s unique geographical features and, in particular, its 
extensive insular territory, the country’s territorial and social cohesion depends 
directly on the existence of frequent and reliable coastal shipping services 
(serving 94 islands, 144 ports and around 36 million passengers per year) 
(European Commission Study, 2013). More information about maritime transport 
can be found in Table 1. 
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Table 1. Information about maritime transport (From European Commission 
Study, 2013). 





- 3996 ships in controlled fleet (>1 000 GT, ‘09) | - Important container ports (contaer throughput im 
| TEU, ‘07): Piraeus: 1 373 000; Thessaloniki: 
~ 969 ships under Greek flag (21 000 GT, ‘09) - a ere errs 


~ Femry passengers (cabotage 09): 64 648 474 - Important cargo ports (metric tonnes, ‘07): 


— Largest merchant fleet in EU and in the world: Piraeus: 20 122 000; Thessaloniki: § 466 000 
o 23.48% of EU fleet n gt (‘08) - Seabome passenger transport between ports (°07): 
45 858 000 


13% of the world’s gross tonnage (Greek- 
owned fleet, ’09) 











This study focuses only on the Aegean Sea area because, as 
demonstrated above, it is more important than the other seas from the 
perspective of shipping routes, number of islands and maritime transport. In 
addition, this area is more vulnerable to illegal activities, as the next section 


describes. 


B. PROTECTING NATIONAL SECURITY 


Greece is facing a major challenge to cope with both the large influx of 
mixed migratory flows, including irregular migrants, refugees and asylum 
seekers, and the current economic crisis. PACE, the Council of Europe’s 
Parliamentary Assembly, adopted Resolution 1918 in 2013 calling for “firm and 
urgent measures [to] tackle the mounting pressure and tension over asylum and 
irregular migration into Greece, Turkey and other Mediterranean countries” 
(European Council, 2013). According to this Resolution, Greece has become the 
main entry point for irregular migratory flows into the European Union, while 


Turkey has become the main country of transit. 


According to statistics provided by the United Nations High Commissioner 


for Refugees, in 2010 more than 132,000 third-country nationals were arrested in 
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Greece, including 53,000 in the Greek-Turkish border regions. During the first ten 
months of 2012, over 70,000 arrests occurred, including about 32,000 at the 
borders of Turkey. People came from 110 different countries, the majority from 
Asia and the Middle-East. The irregular migration routes in the Mediterranean 
Sea, based on 2012 data, are shown in Figure 4. 
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Figure 4. 2012 Map of irregular migration routes in Mediterranean Sea (From 
Interactive map on migration, http://www.imap-migration.org). 


The problem has been traditionally more severe in the Eastern 
Mediterranean than in other regions, and this fact increases the responsibility of 
the adjacent countries to be more effective in monitoring their area of interest. 
The detection of illegal border-crossings in the Mediterranean Sea is shown in 
Figure 5. Although the situation on the Western route is not alarming, on the 
Eastern route detections have followed a remarkably seasonal pattern over the 
last two years with a constantly higher degree of illegal activity. In the second 
quarter of 2012, there were 14,125 detections of illegal border-crossing on the 
Eastern Mediterranean route, an increase of 27 percent compared to the same 
period in 2011. Concerning the Central Mediterranean route, we notice an 
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increase of detections of illegal border-crossings during 2011 due to the turbulent 


sociopolitical developments in North Africa known as the Arab Spring 
(FRONTEX, 2012). 
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Figure 5. Detections of illegal border-crossing between BCPs by main irregular 
migration routes (From FRONTEX, 2012). 


Most migrants and asylum seekers use the Greek area as an intermediate 
step towards Europe. Many of them cannot exit the country due to border checks 
and arrests, and this restriction is one reason many of them return to their 
country of origin. The serious economic crisis of Greece aggravates the situation 
and negatively affects the efforts of the Greek authorities to respond to the large 
influx adequately (Strik, 2013). 


The Greek Government has taken a number of measures to address the 
problem involving the land borders between Greece and Turkey, and these 
efforts have resulted in a tremendous decrease of irregular land border crossing. 
However, this fact impelled illegal migrants to use of sea routes between the two 
countries. During the last months, many reports show that migrants have been 
arriving on Greek islands of the Central and South Aegean Sea. Between August 


and December 2012, 3,280 persons were arrested after crossing the Greek- 
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Turkish sea border as compared to 65 persons in the first seven months of 2012. 
Moreover, an increase of deaths at sea has been confirmed. In September 2012, 
60 people perished when their boat sank off the coast in Izmir, whereas on 15 
December 2012, 20 bodies were found at sea off the coast of Lesvos (Reuters, 
2012). 


Not all of these migrants are individuals seeking asylum. Many cases of 
irregular border-crossing have involved human trafficking. In addition, smugglers 
might also be involved in cases of drugs and weapons trafficking, especially after 
the serious instability in the area since 2012, caused by the political problems in 
Syria. Surveillance capabilities should be enhanced not only to curtail these 
illegal activities but also to improve the legal fishery enforcement and the reaction 


time in Search-and-Rescue (SAR) operations. 


C. CONCEPT OF OPERATIONS 


As stated in the previous chapter, there are a lot of different sensors that 
contribute to the RMP which is defined as “a composite picture of activity of a 
maritime area of interest for a given time” (Dore et al., 2002). To build a coherent 
RMP, a large number of civilian, military, and allied sensor systems must 
collaborate in a systematic way to gather as much information as is possible 
about the activities of the targets existing in the AOI. This information will be used 
by the decision makers at the operational and tactical level to plan and assign 
their assets accordingly. 


The RMP helps to provide commanders with a Common Operational 
Picture (COP). The Joint Publication (JP) 3-0 defines the COP as “a single 
identical display of relevant information shared by more than one command that 
facilitates collaborative planning and assists all echelons to achieve situational 
awareness.” In other words, RMP assists the commanders in obtaining the 
intelligence needed to use their assets more effectively and, consequently, 


achieve their mission. The purpose of the COP is to provide real-time information 
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on Blue, Red, and White forces (friendly, enemy, and neutral vessels) operating 


in the maritime domain. 


Finally, the goal is to improve the Maritime Domain Awareness (MDA), 
which is defined in the JP 3-32 as the “effective understanding of anything 
associated with the maritime domain that could impact the security, safety, 
economy, or environment of a nation.” To achieve complete MDA, all mission or 
functional areas of the components and agencies tasked must be incorporated 
into a common architecture that provides each with the ability to share near real- 
time information, synthesize inputs from multiple sources, and quickly analyze 
the data to affect improved decision making before the opportunity to investigate 
and act on identified threats is lost. 


To achieve the desired detection of enemy combatants, the intelligence 
community utilizes an assortment of Intelligence, Surveillance, and 
Reconnaissance (ISR) sensors. According to JP 1-02, ISR is “an activity that 
synchronizes and integrates the planning and operation of sensors and assets, 
as well as the processing, exploitation, and dissemination of information in direct 
support of current and future operations.” The JP 1-02 further defines the three 
ISR components individually: 


e Intelligence—the product resulting from the collection, processing, 
integration, evaluation, analysis, and interpretation of available 
information concerning foreign nations, hostile or potentially hostile 
forces or elements, or areas of actual or potential operations. 


e Surveillance—the systematic observation of aerospace, surface, or 
subsurface areas, places, persons, or things, by visual, aural, 
electronic, photographic, or other means. 


° Reconnaissance—a mission undertaken to obtain, by visual 
observation or other detection methods, information about the 
activities and resources of an enemy or adversary. 


Intelligence is broader term, while surveillance refers to systematic and 
constant observation of an area or a target, and reconnaissance refers to 
activities performed to obtain all the information needed about a detected contact 
(Brown & Schulz, 2009). 
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D. AGENCIES INVOLVED IN CURRENT SURVEILLANCE SYSTEM 


In this section, the agencies that currently participate in the Surveillance 
System of the Aegean Sea are briefly presented. The Surveillance System of the 
Aegean Sea consists of sensors belonging to different organizations. Because of 
the number of organizations and sensors involved, it is challenging to achieve 
completely accurate and timely information. The stakeholders that currently 


contribute to the National Surveillance System include the following: 


1. Hellenic Navy 


Hellenic Navy (HN) is the main surveillance authority in the Maritime 
Greek territory. HN deploys its units taking into consideration the information 
from the other users (discussed later) and the Operational Plan the leaders draw. 
The ships (including frigates, fast patrol boats and submarines) are equipped 
with the newest devices with impressive capabilities. Although various types of 
ships might be involved in ISR Operations, a typical asset is the Gun Boat 
Osprey HSY 56A (Figure 6). 





Figure 6. HS AITTITOS (P- 268) (From Hellenic Navy, www.hellenicnavy.gr). 
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The navigation radar, the tracking radar and the infrared cameras provide 
advanced detection capabilities, while the radar’s increased range capability is 
an advantage when there is need for covering large areas. The combination of its 
small draft with its relatively large displacement makes this gun boat capable of 
operating in shallow waters and in adverse weather conditions. The drawback is 
the Osprey’s weakness in achieving high speed (>35 kn), which makes it 
incapable of pursuing fast targets. The general characteristics of this type of ship 
are depicted in Table 2. 


Table 2. General characteristics of HS MACHITIS (Type Osprey HSY 56A). 


Length Overall 56.5 m 
Beam (max.) 10m 




















Draft 3.6m 

Displacement 575 tons 

Speed 22 knots 

Range 2,000 miles 

Engines 2 Wartsila Diesel 16 V 25 2 X 5000 
HP 

Weapons 1 OTOMELARA 3" (76mm)/62 


1 OTOBREDA 40mm/70 

2 RHEIMENTALL 20mm 

1 STINGER 

Devices/Systems | LIROD MK 2/ TV CAMERA (FC) 
MIRADOR TVT-IRT Camera, Laser 
Firing Mode 

VARIANT Air-Surface Radar 
BridgeMaster E (Decca) NAVRAD 
RL80C MARPA (Raytheon) 
NAVRAD 

ESM / DR 3000 SLW 

TACTICOS (SMCS) / LINK 11 

















Additionally, the HN controls a number of ground radars located on 
several islands in Central and Eastern Aegean Sea, operated by military 
personnel. These radars collaborate with other sensors (i.e., radars, warships, 
merchant ships, etc.) in the adjacent areas to create the RMP. Although they 


achieve increased detection ranges, these radar systems do not have a 360- 
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degree Field of View (FOV) due to various physical obstacles (i.e., higher hills, 
mountains) resulting in the creation of “blind sectors.” The ground radar system’s 
location is chosen taking into consideration a number of factors, such as what 
area is more important to cover, what elevation provides higher detection range, 


what locations are capable of accommodating these stations and others. 


Additionally, until recently, the HN operated a fleet of Maritime Patrol 
Aircraft (MPA) Orion P-3B, but due to economic deficiencies, they are not 
currently active. The MPA’s missions were mostly in surveillance and anti- 
submarine warfare. These units were capable of flying at 9,000m, and their 


operational endurance was about 10 hours. 


The HN also has the responsibility of exchanging information with 
numerous International stakeholders (NATO, Partnership for Peace countries 
and the EU) as will be explained later in this chapter. 


2. Hellenic Coast Guard 


The Hellenic Coast Guard is a paramilitary organization that can support 
the Hellenic Navy in wartime, but it resides under the control of the Ministry of 
Merchant Marine and Aegean Sea in times of peace. The force has 158 patrol 
craft of various types and four light aircraft. The Coast Guard patrols all Greek 
harbors, coastlines, and territorial waters, monitoring anti-pollution measures and 
controlling merchant shipping. The Hellenic Coast Guard has permanently 
deployed small patrol ships in the major islands of the Central and Eastern 
Aegean Sea, and their task is mostly to deter smuggling and illegal immigration, 
and to perform SAR operations. A typical Coast Guard vessel that is used to 
patrol is the Panther 57 (Lambro57) (Figure 7), which is capable of achieving 
speeds over 40 kn. Its shallow draft and protective water jet intake grill make it 
capable of operating in waters as shallow as one meter deep. In contrast, it is 
equipped with only navigation radar and night vision binoculars, and due to small 
tonnage, it is extremely vulnerable to weather conditions. The general 
characteristics of this type of ship are depicted in Table 3. 
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Figure 7. Hellenic Coast Guard’s typical patrol boat (Panther 57) (From 
MotoMarine shipyards, www.motomarine.gr). 


Table 3. General characteristics of Panther 57 


Length Overall | 18.2 m 


| Beam (max.) | 4.68m 
| Draft | 0.92 m 
| Displacement | 28 tons 
| Max Speed | 44 knots 


























| Range | 250 miles 

| Engines | 2 Engine Type 12V Man 

| Weapons | 0,50 Browning Automatic Cannon 
3. Hellenic Air Force 


Surface Surveillance is not among the tasks of the Hellenic Air Force. 
Nevertheless, use of RF-4 aircraft can contribute to reconnaissance, especially 
during wartime or SAR operations in peacetime. 


4. Hellenic Army 


The large number of islands in the Aegean Sea, either big habitable ones 
or small islets, forms a rare environment for operations. Being on an island one 


may have a view of the neighboring island(s) most of the time, and this allows a 
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person to obtain a clear understanding of the interim sea space. Trying to take 
advantage of this spatial uniqueness, the Hellenic Army has located a number of 
coastal (sea) and Ground Surveillance Radar BOR-A550 (Figure 8) suitable for 
army, border/coast guard and security applications. It combines surveillance of 
ground, sea and lower level airspace in a single radar system whereby moving 
targets will be detected, located, automatically classified and tracked day and 


night under severe weather conditions. 


Furthermore, across the Greek coastline of the Eastern Aegean Sea 
islands, there are a large number of sites with observers equipped with basic 
vision tools (i.e., binoculars, night vision goggles). All the information about 
targets in the AOI is passed to the National Surveillance System so that all the 


stakeholders are aware of the RMP. 





Figure 8. Radar BOR-A 550 (From Hellenic Army, www.army.gr). 


5. NATO 


The Maritime Command and Control Information System (MCCIS) is a 
military maritime command and control system that has been developed and 
maintained for members of NATO. The MCCIS assists strategic and tactical 
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commanders and their staffs in the decision making process. MCCIS is installed 
and in operational use at 61 sites in different countries, one of which is Greece. 
There are over 250 users. All the member states are encouraged to contribute to 
the RMP, and they also have the privilege of sharing all the data without 


censorship. 


MCCIS electronically processes data from multiple sources, displays data 
in various command and control applications, and allows the user to manipulate 
this data. One important component in NATO’s MDA is NATO’s Maritime Safety 
and Security Information System (MSSIS), which is based around the acquisition 
and analysis of Automatic Identification System (AIS) data. This data is gathered 
from the AIS systems of NATO member states as well as by a number of other 


non-NATO states on the basis of bilateral agreements. 


Once the MSSIS analysis is complete, the relevant data is fed into 
NATO’s MCCIS, as shown in Figure 9. Data held within the MCCIS is classified 
as, in addition to containing the compiled picture created through MSSIS, it also 
includes intelligence data, classified surveillance data (from satellites and other 
sensors) and the real time location of NATO assets (such as warships). One of 
the reasons why the basic MSSIS is kept at an unclassified level, however, is to 
attract the participation of non-NATO states (European Commission, 2008). 


As regards the acquisition and processing of surveillance data, maritime 
or otherwise, NATO does not have an explicit mandate as such. Instead such 
activities are to be implied from the content of Article 3 of the North Atlantic 
Treaty, which states: “In order more effectively to achieve the objectives of this 
Treaty, the Parties, separately and jointly, by means of continuous and effective 
self-help and mutual aid, will maintain and develop their individual and collective 


capacity to resist armed attack.” 
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Figure 9. Diagram of MSSIS (From European Commission, 2008). 


6. European Union 


In 2004, the European Agency for the Management of Operational 
Cooperation at the External Borders of the Member States of the European 
Union (FRONTEX) was established, and its Eastern Sea Borders Centre is 
located in Piraeus, Greece. FRONTEX’s mission is to help EU member states 
implement EU rules on external border controls and to coordinate operational 


cooperation between member states in the field of external border management. 


While it remains the task of each member state to control its own borders, 
the agency is vested with the function of ensuring that they all do so with the 
same high standard of efficiency. To reduce duplicated effort and hence save 
time, money and other resources, the European Patrols Network was born in 
2007. Greece, as guardian of the Eastern borders, has been assisted periodically 


by other EU countries with the deployment of a few patrol ships and aircraft. 


The EU also has developed SafeSeaNet, a vessel traffic monitoring and 
information system, established to enhance maritime safety, port and maritime 
security, marine environment protection and efficiency of maritime traffic and 
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maritime transport. It was established as a centralized European platform to 
facilitate the maritime information sharing between European countries. The main 
sources of information consist of the AIS reports and notification messages sent 


by designated authorities in participating countries. 


On February 13, 2008 the European Commission adopted a 
Communication on the creation of a European Border Surveillance System. It 
was designed to support the member states in their efforts to reduce the number 
of illegal immigrants entering the European Union by improving their situational 
awareness at their external borders and increasing the reaction capability of their 
information and border control authorities (Commission of the European 
Communities, 2008). It also focuses on the EU's southern and eastern maritime 


borders. 


E. AIRBORNE PLATFORMS 


Unmanned aircraft are commonly called unmanned aerial vehicles 
(UAVs), and when combined with ground control stations and data links, they 
form a UAS, or unmanned aerial system. Conventional wisdom states that the 
UAS offers two main advantages over manned aircraft: they are considered more 
cost-effective, and they minimize the risk to a pilot’s life (U.S. Library of 
Congress, 2011). Because this study attempts to investigate the ways in which a 
UAS can be integrated into the current surveillance system, its basic applications 
and the major systems which might be used are being discussed in this section. 


JP 3-52 defines a UAS as ‘that system whose components include the 
necessary equipment, network, and personnel to control an unmanned aircraft.” 
According to this publication, the term UAS will be used instead of UAV across 
the U.S. Department of Defense. In this study the UAS Program of U.S. Navy is 
used as a reference, but the terms UAVs and UAS are used interchangeably. 
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1. Applications for UAVs 


From the U.S. Navy’s perspective, UAVs are currently being used in a 
large variety of applications. Of course not every UAV is capable of 
accomplishing all the missions, but each type of UAV is designed to carry 
different payloads and sensors; for this reason, UAVs can be used for a variety of 
applications. Some of the applications that the current UAVs are used for or for 
which they are about to be used in the near future are: penetrating strike, ISR 
operations, ELINT (intelligence derived from electromagnetic radiations from 
foreign sources) Collection, COMINT (technical and intelligence information 
derived from foreign communications) Collection, air-to-air combat, airborne 
electronic attack, suppression of enemy air defenses, close air support, 
chemical-biological-radiological-nuclear detection, battle damage assessments 
(BDA), mine detection, precision target designation, anti-submarine operations, 
suppression of enemy air defenses (SEAD) and psychological operations, such 
as dropping leaflets (Alkire, 2010; U.S. Library of Congress, 2011). 


In a 2004 study, each Combatant Command and Service was asked to 
rank the importance of 18 missions relative to four general classes of UAS (i.e., 
small, tactical, theater, and combat). Their responses are depicted in Table 4. 
Although the distinction between the classes of UAS is not the same across time, 
it is notable that Reconnaissance, what we are most interested in for this study, 


is ranked as the highest priority regardless of the type of platform. 
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Table 4. Combatant commander/service UAS mission prioritization matrix 
(From Office of the Secretary of Defense, 2005). 


Mission Seri Bestel Theater COU ily a JFC 


Recnoneisanee | @ | @ | @ | @ fm 


Signals Intel 10 3 BA 
Mine Detection/CM 7 11 FP 


PB | ow | 
Precision Target Location and 5 : 3 7 FA 
Designation ~ a 2 
Battle Management 4 10 4 7 c2 
CenlioRenmsime | 3 [7 | ¢ |» ff ma 


Counter Cam/Con/Deception 8 5 7 11 BA 
Electronic Warfare 14 9 10 - FP 


cob SAR Ce 
Communications/Data Relay | os | 6 | 5s | 8s | C2 
Information Warfare 15 12 6 FA 
Digital Mapping 11 13 — 12 BA 
Littoral Undersea Warfare 17 15 | wm | | FA 
SoFTentemy [9 | | | 























Weaponization/Strike 16 4 12 3 FA 
GPS Psuedolite 18 18 15 18 GZ 
Covert Sensor Insertion 12 14 | 16 | 1s [ff BA 
Decoy/Pathfinder 13 17 18 17 FA 








Additionally, UAVs can provide critical support for SAR operations. 
However, to achieve their full potential, all the factors that affect the UAV’s 
performance, such as payload capabilities, endurance, communications and 
environmental hazards, should be taken into consideration (Waharte & Trigoni, 
1997). 


Moreover, in this paper we are more interested in tasks that UASs can 
assume in the current framework of the HN. Therefore, in this section, traditional 
navy missions are emphasized rather than naval contributions to irregular 


warfare. 
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2. Types of UAVs 


UAVs can be divided into many categories taking into consideration 
various factors like the missions they accomplish, their characteristics, the 
platform from which they are launched or their cost. According to the National 
Research Council (2005), the Navy views its future use of unmanned aerial 
vehicles to be primarily in three categories: 


° Long-dwell, standoff intelligence, surveillance, and reconnaissance 
(ISR), as exemplified by the Broad Area Maritime Surveillance 
(BAMS) concept and the Global Hawk Maritime Demonstration 
(GHMD); 


° Carrier-based, penetrating surveillance and SEAD/strike Joint 
Unmanned Combat Air System ; and 


° Ship-based tactical surveillance and targeting, which call for a 
vertical takeoff- and-landing system that can operate from a variety 
of ship types. 


As stated earlier, the U. S. Navy is making large investments in a number 
of major UAV programs, including BAMS, the Unmanned Combat Aircraft 
System Demonstrator, the Fire Scout vertical takeoff/landing tactical UAS 
(VTUAS), and the Small Tactical/Tier Il UAS (STUAS/Tier II UAS) (Alkire, 2010). 
In general, UAVs can be launched from several platforms like shore bases or 
infrastructures, from surface ships or from aircraft carriers. Because the latter has 
no implementation in the HN, these vehicles are not studied in this paper. In the 
following section, we separate UASs into those categories that are of relevance 
to the HN and briefly present their main characteristics. 


a. Broad Area Maritime Surveillance (BAMS) 


The U.S. Navy is developing the BAMS unmanned system to 
provide a persistent, maritime, worldwide access, ISR capability (Figure 10). In 
FY 2003, the U.S. Navy purchased two RQ-4A Global Hawk variants with 
electro-optical/infrared (EO/IR) and SAR sensors, along with their ground control 
stations and support equipment. These are known as the GHMD System. It is a 
high-altitude, long-endurance (HALE) Unmanned Aerial Surveillance System and 
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provides the U.S. Navy with demonstration capability primarily for doctrine, 
concept of operations, and tactics, techniques, and procedures development. 
The BAMS UAS, named RQ-4N, is a maritime derivative of the Global Hawk 
equipped with Navy-specific control stations called Tactical Control Systems. It 
will have a full 360-degree field of regard and the capability to collect full motion 
video. It also requires a runway for takeoff and landing and is not carrier-capable 
(Alkire, 2010). 
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Figure 10. Graphical representation of BAMS Operations (From Northrop 
Grumman, www.northropgrumman.com). 


The aircraft has a projected 12,000 nm range and 35-hour 
endurance, with satellite and line-of-sight communication links to the ground 
system. High resolution sensors that can look through adverse weather, day or 
night, from an altitude of 65,000 feet, can conduct limited maritime surveillance 
over an area the size of the state of Illinois in only 24 hours (Northrop Grumman, 


2007). Detailed specifications are shown in Table 5. 
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Table 5. 


Specifications of RQ-4 Global Hawk Maritime Demonstration 


System (From Northrop Grumman, www.northropgrumman.com). 



































Wingspan 116.2 ft (35.4 m) Height 14.6 ft (4.2 m) 
Gross 
Length 44.4 ft (13.5 m) Takeoff nee 
Weight wee 
Maximum 
Payload 2,000 Ibs (907.2 kg) | Altitude 65,000 ft (19.8 km) 
10,000 nm Loiter 
Ferry Range (18.520 km) Velocity 343 knots 
On-Station Macimiun 
Endurance at 24 Hours =Aaurane 35 Hours 
1,200 nm ce 
Synthetic ; Maritime 
Aperture Radar rrr peau Target Cue for ISAR 
(SAR) P Acquisition 
Maritime : ; 
Search 15,000 sq. km/Min ISAR 3 Resolutions 
NIIRS 6.0/6.5 NIIRS 5.0/5.5 
Electro-Optical (WAS/Spot) Infrared (WAS/Spot) 





b. 


Vertical Takeoff/ Landing Tactical Unmanned Aerial 


Systems 





The Fire Scout VTUAV system provides remarkable situation 
awareness and precision targeting support. The MQ-8B Fire Scout (Figure 11) 
has the ability to take off and land on any aviation-capable warship autonomously 
and at appropriate land areas without any special infrastructure. 





Figure 11. MQ-8B Fire Scout (From America’s Navy, www.navy.mil). 


The Fire Scout VTUAS has an operational footprint that is a fraction 
of that of the multipurpose MH-60-class helicopters; it can operate from, and 
provide the UAS advantages to, surface ship platforms. The U.S. Navy has 
decided to replace helicopters with VTUAVs during the evolution of the Littoral 
Combat Ship program (Burgess, 2004). This type of UAV has been long tested 
and used by the U.S. Armed Forces. In 2009, 110 ship take-offs and landings 
were conducted, and during this time period the Fire Scout completed successful 
deployments in real combat fields (Jacobson, 2010). 


The Fire Scout has an operating ceiling of 20,000 ft and a total 
endurance of over eight hours that provides more than six hours on station with a 
standard payload at 110 nm (200 km) from the launch site. A system of two Fire 
Scouts can provide continuous coverage at 110 nm. Utilizing a payload that 
includes EO/IR sensor with laser rangefinder/illuminator and maritime radar, the 
Fire Scout can find and identify tactical targets, track and illuminate them (Figure 
12) and accurately provide targeting data to strike platforms (Northrop Grumman, 
2012). Detailed characteristics are shown in Table 6. 
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Figure 12. Images taken by MQ-8B Fire Scout’s EO/IR sensors (From Northrop 
Grumman, www.NorthropGrumman.com). 


Table 6. Specifications of MQ-8B Fire Scout (From Northrop Grumman, 
www.NorthropGrumman.com). 















































Fuselage Length 23.95 ft (7.3 m) 

(with Dual Payload Nose) 

Fuselage Width 6.20 ft (1.9 m) 

Length 30.03 ft (9.2 m) 

(with Blades Folded Forward) 

Rotor Diameter 27.50 ft (8.4 m) 

Height (Top of Tail Antenna) 9.71ft (2.9 m) 

Gross Weight 3,150 Ibs (1428.8 kg) 

Engine Rolls Royce 250-C20W 
Turboshaft Engine 

Speed 115+ Knots 

Ceiling 20,000 ft (6.1 km) 

Total Flight Time with Baseline Payload | 8+ Hrs 

Total Flight Time with EO/IR + Radar 7+ Hrs 

Total Flight Time with Maximum Payload | 5+ Hrs 

Payloads EO/IR/LRF/Mine 
Detector/Comm.Relay/ Maritime 
Radar 
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The MQ-8B Fire Scout VTUAV is used to recapitalize the capability 
of the aging fleet of P-3 Orion aircraft and provide maritime domain awareness 
for the U.S. Navy (Alkire, 2010). The Fire Scout can be used in various missions, 
such as SSC, Birddog/tattletale operations, Maritime Interdiction Operations, 
BDA, ISR operations and Target Acquisition/Anti-Submarine Warfare, border 
patrol, SAR operations and medical resupply (Berner, 2004). 


C: Small Tactical/Tier I! Unmanned Aerial Systems (STUAS) 


In 2005, the U.S. Navy signed a $14.5 million contract with Boeing 
to provide ISR coverage with the Scan Eagle Small Tactical UAS (STUAS) 
(Alkire, 2010). In 2009, a Scan Eagle UAS was operated from a nearby U.S. 
Navy vessel to provide real-time situational awareness during a Somali pirate 
incident that ultimately ended with the safe release of the captain of a U.S. cargo 
ship. Following the successful tests and in-the-field support, in July 2010 the 
Department of the Navy awarded Insitu a two-year, $43.7 million contract for the 
design, development, integration and test of the STUAS Integrator RQ-21A for 
use by the Navy and Marine Corps (U.S. Library of Congress, 2012). 


For the U.S. Navy, the RQ-21A (Figure 13) will provide persistent 
Reconnaissance, Surveillance, and Target Acquisition support for tactical 
maneuver decisions and unit-level force defense/force protection for Navy ships, 
Marine Corps land forces, Navy Expeditionary Combat Command forces and 
Navy Special Warfare Units (Small tactical unmanned aircraft system, n.d.). 
According to the manufacturer, upon completion of the project the RQ-21A will be 
able to handle the following missions: Search and Rescue, Disaster Response, 
Force Protection, Combined Arms, Target Following, Battle Damage 
Assessment, Pattern of Life, Border Security, Asset Protection, Wildlife 
Monitoring, Agricultural Assessment, Communications Relay, Networked 
Operations and Anti-Piracy (Insitu, 2012). 


35 





Figure 13. The Integrator RQ-21A is recovered with Insitu's SkyHook capture rope 
after its first operational take offs and landings at sea from the San 
Antonio class dock landing ship USS Mesa Verde (LPD-19) in the Gulf 
of Mexico on 10 February 2013 (From Naval Drones, 
www.navaldrones.com). 


The RQ-21A will consist of a number of Air Vehicles (AVs), Ground 
Control Systems (GCS) and multi-mission payloads, which will provide 
intelligence coverage, surveillance, reconnaissance, and communications relay 
for up to 15 hours per day continuously with a short surge capability for 24 hours 
a day. Payloads include Day/Night Full Motion Video cameras, infrared marker, 
laser range finder and AIS receivers. Ancillary equipment includes 
launch/recovery mechanisms, tactical communications equipment and spares. 
The RQ-21A will have a minimal operating radius of 50 nm, and the AV will be 
capable of airspeeds up to 80 nm per hour (knots) with a service ceiling of 
15,000 ft density altitude. The fully autonomous launch and recovery system will 
require minimal space for takeoff and recovery from an _ unimproved 
expeditionary/urban environment, as well as from the deck of Navy ships. Initial 


operational capability is planned for 2013 (Small tactical unmanned aircraft 
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system, n.d.). Characteristics of UAS Integrator (RQ-21A) are depicted in 
Table 7. 





























Table 7. Specifications of Integrator (RQ-21A) (Insitu, 2012 ). 

Wingspan: 16 ft/4.8m 

Length: 7.2 ft/2.2m 

Empty structure weight: 80 Ib / 34.0 kg 

Max takeoff weight: 135 Ib / 61.2 kg 

Max horizontal speed: 80+ knots / 41.2+ m/s 

Cruise speed: 55 knots / 28.3 m/s 

Ceiling >15,000 ft / 4,573 m 

Endurance: 24 hours 

Payloads Electro-optic, Mid-wave infrared, IR 
marker*, Laser rangefinder* 
* Class 3B laser product. 











F. DETECTION PROBLEM AND RELATED STUDIES 


This section briefly discusses the problem of detection, which involves the 
major issues that impede the detection and identification procedure, as well as 
the most important studies made on surveillance from UAVs either on the ground 


or in the maritime domain. 


1. Detection Problem 


The detection and identification problem, as one can easily understand, is 
very complex as it involves a variety of sensors and agencies. The sensors 
involved are mostly Radio Detection and Ranging (RADAR or radar) systems 
and Electro-optical/Infrared (EO/IO) sensors. Radar systems are active sensors 
that emit electromagnetic waves which are reflected from targets and then 
received back by their sensors. Compared to EO/IR systems, radar systems 
have almost the same performance during day or night under clear weather, but 
radar systems suffer less atmospheric attenuation than EO/IR systems. For this 


reason, they are able to detect targets at greater ranges. 


Conversely, EO/IR systems are passive devices and work with the 
radiations emitted by the target. EO sensors provide visual images of the targets 
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and are mostly used during day. However, they are limited by any factor (such as 
clouds, smog or camouflage) that hampers visual contact between the sensor 
and the target. IR cameras transform the thermal energy that the targets emit to 
a video signal. The drawback of IR sensors is that they cannot provide accurate 
image details of the target, but an experienced user can easily determine the 
type of target from the resulting image. In the field, units with EO/IR sensors are 
used to identify targets that have already been detected by other sensors (radar, 
other reports, intelligence, etc.) because in that way the target information is 
received in less time (Brown & Schulz, 2009). 


Taking into consideration that UAVs are based on the EO/IR that they 
carry, sea state and adverse weather conditions, such as rain droplets, snow, fog 
and hail stones, will all affect their operation. Moreover, limited endurance or 
incapability to operate in the aforementioned weather conditions may influence 
the UAV’s performance. These conditions may affect the unmanned vehicle itself 
or the platform from which it is launched (Johnson, 2004). 


Perhaps the most important factor that affects the IR performance is rain. 
Precipitation causes attenuation of the signal (Figure 14) and also could result in 
a loss of communication between the GCS and the UAV because radio 
frequencies are often used for the communication channel. The impact of surface 
clutter diminishes EO/IR sensor effectiveness, as well (Johnson, 2004). 
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Figure 14. Specific attenuation for rain calculated for two model drop size 
distributions (From Crane, 1981). 


The target resolution of a radar system has the ability to distinguish 
between targets that are very close in either range or bearing. Weapons-control 
radar, which requires great precision, should be able to distinguish between 
targets that are only yards apart. Search radar is usually less precise and only 
distinguishes between targets that are hundreds of yards or even miles apart. 
Resolution is usually divided into two categories: range resolution and bearing 


resolution. 


Range resolution is the ability of a radar system to distinguish between 
two or more targets on the same bearing but at different ranges (Figure 15). The 
degree of range resolution depends on the width of the transmitted pulse, the 
types and sizes of targets and the efficiency of the receiver and indicator. Pulse 


width is the primary factor in range resolution. 
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Figure 15. Range Resolution. The left pair of targets in closer distance is 
detected, but the right one at a greater distance is not (From Radar 
Tutorial, www.radartutorial.eu). 


2. Related Studies 


In this section, we provide an overview of studies that investigated how 
the type of UAVs used can affect their tactics. For example, the payload a UAV 
can carry determines the way it will be used. If a UAV has long endurance, it will 
be assigned to cover large areas; if this is not the case, then operational priorities 
will have to be set. Furthermore, the platforms from which they are launched 
along with the extent to which they are affected by adverse weather conditions 
are crucial for operational or tactical commanders to take into consideration. In 
addition, the number of UAVs needed to complete the identification mission is 
dependent upon the size of the search area and the sweep-width of a UAV’s 
sensors (Washburn, 2002). 


Berner (2004) studied the effective use of multiple UAVs for the Navy’s 
SSC Mission. More precisely, he studied how a BAMS UAV and VTUAVs 
working together can provide increased situational awareness in the maritime 


environment. The best UAV combination is BAMS plus two or three VTUAVs. 
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However, Berner’s analysis showed that small numbers of VTUAVs can perform 
as well without BAMS as they do with BAMS. For combinations with multiple 
UAVs, BAMS proves to be a valuable asset that not only reduces the number of 
missed classifications, but greatly improves the amount of coverage on all 
contacts in the maritime environment. BAMS tactics have less effect than the 


mere presence of BAMS itself. 


Lalis (2007), using agent-based simulation (MANA), identified that the 
most important factors when planning UAV operations are the size of the area 
being covered, the Time on Task (ToT) and the detection range of the UAV. He 
also assumed that the UAV’s probability of detection is perfect (Probability of 
Detection=1) when a target is within a radius of 2-18 nm (“cookie-cutter” 
approach). This led to the conclusion that the length of the shoreline has no 
impact on the target’s survivability because UAVs fly over the whole coastline. 
Lalis also studied the effect of the UAV’s three search patterns on target 
detection and concluded that “if ToT is not under consideration, then the three 
patterns are not different." Conversely, other study suggests that “the patterns 
that UAVs fly have a direct effect on the coverage area and probability of 
detection of contacts of interest” (Gottfried, 2004). 


McMindes (2005) studied the impact of a wide variety of factors, such as 
UAV speed, stealth, altitude and sensor range, as well as enemy force sensor 
ranges, probability of kill, array of forces and numerical strength on the UAV’s 
survivability. He concluded that a speed between 135 and 225 kn increases the 
UAV’s survivability. The exception to speed’s dominance is in the face of 
extremely high capability enemy assets. In this case, stealth becomes more 
important than speed alone. However, the interactions indicate that as both 
speed and stealth increase, speed yields a faster return on overall survivability, 
and that speed mitigates increased enemy capabilities. Concerning altitude, 
increased altitude produces higher mean survivability as well as decreased 


variability. 
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Yildiz (2009) studied the use of mini-UAVs in border security and found 
them to be beneficial in capturing the illegal entrants and thus could potentially 
provide more secure borders. Adequate manpower and a reliable communication 


scheme to compose a COP emerged as the most important factors. 


G. TASK ANALYSIS 


EO/IR systems produce images that must be scanned for recognizable 
patterns of targets. Computers have an ability to scan imagery continuously for 
long periods of time without risk of degraded performance due to boredom or 
fatigue. Humans, on the other hand, are characterized by a greater capability to 
pattern match visual images, but they are susceptible to performance errors. 
Combining computer and human analysis allows for the benefits of each 
resource to be utilized resulting in enhanced capabilities. The analysis of EO/IR 
images can be done at the location of the sensor or at a central location (Brown 
& Schulz, 2009). 


In this section we conduct a task analysis, because it is the process that 
describes the user’s task in detail and helps stakeholders to design a system that 
supports users in doing this task in the most effective way. Task analysis has 
long been used as a fundamental step in system design and is useful for 
examining new or existing systems. Understanding the process and task 
structure of target detection may shed more light on how surveillance can be 


conducted. 


Because the surveillance task used in this study is very complex and 
incorporates inputs from and activities performed by many field users, we 
subdivide it into four distinct subtasks. These subtasks are: communications, 
search, detection and identification. The subdivision of a system into autonomous 
tasks is, in general, a powerful technique for studying a complex system. In the 
case of distributed surveillance systems, the subdivision of a system into 
elementary modules is necessary for the distribution of intelligence (Marcerano 
et. al, 2001). 


42 


Note that the following task analysis (Table 8) refers to a high level of 


command because too many details would be out of the scope of this study. 


Table 8. Maritime surveillance task analysis. 





No Task Subtask/ user action System Actions 

1 1.1 Operate the appropriate Telephones, VHF, UHF, 
communication Link 11-14 

devices 


1.2 Get familiar with the 
functionalities 
and the military 
procedures 

1.3 Take measures against 


information 
leakage 













Implement corresponding 
doctrines 


1.4 Set up communication Set up communication 
with other with your 
























partners collaborators 
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2.1 Gather information 


concerning 
weather 
conditions 





Sea state, visibility, wind 
direction & 
strength, rain 





2.2 Allocate the units to 
cover all the 
area of 
responsibility 





2.3 Take into consideration 
information for 
potential targets 
in the area 


2.4 Define the search pattern 
for the 
subordinate 
units 


2.5 Approach the shoreline 
to detect 
potential targets 
which electronic 
devices cannot 
detect due to 
technical 
restrictions 


2.6 Eliminate blind sectors 


2.7 Search for electronic 
emissions (if 
available) 

2.8 Minimize the depth of 


overlapping 
sectors 





2.9 Report unit position 
according to the 
procedures 





44 


If this is not feasible, give 
priorities 


Use the information to 
conduct 
search more 
intensely 





Search an area/search 
around a 
specific 
position 


Being closer to the 
shoreline the 
radar 
resolution is 
increased 








IFF, monitor AIS system, 
ESM 


Searching in adjacent 
sectors 
should not be 
extensively 
overlapped in 
order to 
reduce the 
cost 

















Detection 


Identification 





3.1 Record time and position of 
the detected target 


3.2 Match the detection with 
information for 
potential target in 
the area 


3.3 Search more thoroughly in 
the area close to it 
for additional 
targets 


3.4 Verify that it is nota 
pseudo-target 


3.5 Report its time and speed 





4.1 Correlate information 
provided by other 
agencies to identify 
the detected target 


Are you expecting a target 
in this area or 
not? 


Approach the target to have 
better visual 
contact and 
increase radar 
resolution 


Tune your device for 
optimum 
performance 





What type of target am | 
expecting in this 
area? 





4.2 |dentify visually the 
detected target 


4.3 Identify the target using 
electronic devices 


4.4 ldentify the target by its 
electronic 
emissions (if 
available) 


Identify positively the target 
using devices 
(electro optical , 
I/R) 


IFF, monitor AIS system, 
ESM 





4.5 Report its identity 


4.6 Keep continuous contact 
with the identified 
target 


4.7 Record its electronic 
emissions in the 
data base (if 
available) 








Enrich the corresponding 
data base for 
future use 
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lil. MODEL DEVELOPMENT 


A. MODEL SELECTION 


Models are used to represent and describe the behavior of the system 
being modeled. Models assist users in gaining a better understanding of the real 
world, and in most cases they present real systems in a simplified way. In 
general, models are the most cost-effective way to study a system. They are also 
a risk-free solution in cases where dealing with the actual system itself might put 
the user in danger. Another benefit of models is their ability to reduce the time, 
space, and means needed to study a system (Sanchez, 2007). 


“A system is defined to be a collection of entities that act and interact 
together toward the accomplishment of some logical end. In practice, what is 
meant by system depends on the objectives of a particular study” (Law, 2007). 
Systems can be classified in a variety of ways. In this study we focus on 
simulation solutions to study the efficacy of using UAVs to monitor a complex 


marine environment. 





Figure 16. Ways to study a system (From Law, 2007). 
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Analytical solutions are preferred when the elements that compose the 
model are relatively simple. Using such an approach, it is possible to obtain a 
closed-form solution and get a precise answer. These methods may involve 
algebra, calculus, probability theory, linear programming, decision analysis, 
Markov chain analysis and queuing theory (Law, 2007, pp. 3-6; Berner, 2004). 
However, when a system is highly complex and an analytical solution is 
infeasible, we use simulation solutions, instead. In a simulation, we use a 
computer to evaluate a model numerically, and data are gathered to estimate the 


desired true characteristics of the model (Law, 2007). 


Simulation has both advantages and disadvantages, and every 
user/analyst should be aware of them. By simulating a system, we numerically 
exercise the model to investigate how the inputs affect the output measures of 
performance. That means that we are capable of making predictions of input 
changes. Additionally, visualization can give a good understating of a system and 
allow us to study its behavior in detail. On the other hand, a simulation, as are all 
models, is limited by its assumptions. Some assumptions may be reasonable, but 
some may be implemented solely to make the model less complex. In any case, 
these assumptions are set by the user, and in altering these assumptions, we 
change the output, yielding a less robust solution. Furthermore, in some cases, it 


can be time consuming to try to build complex models with debatable results. 


B. MAP-AWARE NONUNIFORM AUTOMATA 


In this study, the Map-Aware Non-uniform Automata (MANA) Version V 
software is used as the simulation tool. This section explains the reasons that 


this model was chosen and describes briefly its major features. 


1. Overview of MANA 


MANA is an agent-based, time-stepped simulation modeling environment. 
Agent-based models seem to be the most appropriate for the current study 
because of their ability to describe the behavior of individual entities in complex 


systems. MANA is a Discrete Time Simulation (DTS), which uses a fixed time 
48 


increment as the time advance method (Alrowaei, 2011). DTS is the most 
commonly used time advance mechanism in combat simulation and agent-based 
models (Macal, 2010, pp. 371-382). The fact that MANA also has been used in 
numerous Naval Postgraduate School theses and in other studies in the military 


domain indicates its usefulness. 


MANA has been built upon two key ideas. The first idea describes how the 
model will contain entities that are controlled by decision-making algorithms. 
These entities (agents) interact with each other, as well as the environment in 
which they operate, and make their own decisions based upon the “personality” 
the modeler gave to them. The second idea has to do with the simplicity of the 
model. The creators of MANA claim that more detailed models are not 
necessarily better (McIntosh, 2007). Additionally, the non-linear nature of 
equations describing real situations is sensitive to initial conditions resulting in 
non-robust solutions (McIntosh, 2007). Perhaps, this is the biggest benefit of 
MANA; although it is designed to describe complex systems, like any other 


simulation tool, it does so with comparatively simple models. 


MANA is intended to address a wide spectrum of problems and was 
designed for use in scenario exploring. That is, the modeler can vary the 
parameters of the model, even while it is running, and observe the output. This 
feature is a great benefit for the analysts because they have the capability to 
explore the relative importance of a variable on the final result. Furthermore, 
MANA is user-friendly and allows the modeler to create his/her own scenario in a 
short time using a very simple and understandable interface. Building the 
appropriate scenario, the user can explore how the agents perform and gain 
insights about the potential outcomes. 


2: MANA Characteristics 


In this section, the innovative features of MANA compared to other 
models, along with its basic characteristics, is briefly discussed. The most 


important elements of MANA and its concepts are presented to provide the 
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reader with information most relevant to this study. For further details see 
McIntosh (2007). 


Agent is the individual entity that “operates” in the scenario. Each 
entity has certain personality traits that drive it toward or away from 
other entities, waypoints or areas with specific characteristics (i.e., 
areas that provide cover or areas in which it is easier to move). An 
agent is also equipped with sensors, weapons and speed of 
movement, all of them with capabilities arranged by the modeler. It 
may also have established communication links with other entities. 


Squad is a group of agents of any size, as defined by the user. 
Agents in a squad share the same propensities, the same 
capabilities and the same SA Map. In a naval scenario, like those 
studied in this thesis, a squad may be considered as a squadron of 
ships of the same type. 


Battlefield is the area in which the scenario is taking place. Unlike 
previous versions, in MANA V battlefield distances and agent 
speeds can be specified directly in terms of real-world distances 
and times. The battlefield time interval can be set by the user. The 
shorter the time interval, the more accurate models we have, but 
more calculations are needed resulting in more time consuming 
simulation runs. A background map can be loaded for illustrative 
purposes without any impact on the simulation run. In contrast, 
terrain maps can be used to identify terrain features. These maps 
(Figure 17) are standard Windows bitmap files, which use colors to 
impact the agent movement in the battlefield. Apart from the default 
colors, the user can create various colors to capture terrains with 
different characteristics with respect to three factors: Going, Cover 
and Concealment (Table 9). These factors act as multipliers to 
movement speed, hit rate and sensor detection, respectively. For 
example, if an area has been defined to have Going=0, Cover=1 
and Concealment=1 then the agent who will be there will not be 
able to move, he cannot be shot at, and no one can see him. 
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Figure 17. Background Map (left) used in a scenario in MANA. Terrain Map (right) 
for the same scenario. The values that have been set for the colors in 


the Terrain Map are noted in Table 9. 


























Table 9. Values for Going/Cover/Concealment of the Terrain Map depicted 
in Figure 17. 

Name Going Cover Concealment Color 
Sea 1.00 0.00 0.00 Black 
Land 0.10 0.00 0.00 Grey 

Near Land 0.75 0.50 0.50 Light Green 





Trigger State feature provides MANA with additional flexibility. As 
the scenario evolves, certain events may occur to an agent (or its 
squad). These events might be the presence of an enemy in its 
vicinity, the arrival at a waypoint, a shot taken, an agent injury and 
many more. If an event happens, the modeler can define a new set 
of agent personality weightings, speed, sensor capabilities or any 
other parameter that “describes” this agent. The modeler is able 
also to define the time that the agent shifts in the new “trigger 
state,” as well as a potential sequence of trigger states, enhancing 
the agent’s behavior which might emerge in a scenario. 


Sensors are critical features for the agents. Each entity can carry 
up to six sensors, either simple or advanced. “Simple sensors only 
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provide a cookie-cutter model of detection and classification while 
advanced sensors allow the specification of a range-integration 
time profile for both” (McIntosh, 2007). The term “cookie-cutter” 
means every visible contact within the detection range will be 
detected (or classified for the case of a classification range). On the 
other hand, the advanced sensor type uses times between 
detections with respect to the range solely for the detection task. 
For the classification, it uses range-dependent probability of 
classifying detecting events. It is important to note that detection is 
a prerequisite for classification. 


Situational Awareness (SA) Map is one of the innovative elements 
of MANA. It is graphical representation of a squad’s perception of 
the battlefield. That is, it is a map that depicts the contacts 
(classified or not) of which the squad is aware. It represents intra- 
squad communications whereby squad members actively share 
information, as it would be expected in a real combat team. The 
modeler can also define the time taken for information to get from 
the agents’ sensors to SA Map. This definition models potential 
time delays in information sharing. 


Inorganic SA Map represents information obtained remotely from 
other squads, as opposed to the squad’s own sensors. The 
modeler defines which squads could communicate as well as the 
quality of the communication with respect to several different 
parameters such as Latency, Reliability, Capacity and Filtering of 
information. Figure 18 is a snapshot of an SA Map and an Inorganic 
SA Map for the same time of a simulation run. The main difference 
between the two is that, in the second, only contacts coming in from 
remote squads will be displayed. In reality, the agent is aware of 
both the pictures. With all these features, MANA is an appropriate 
tool to study aspects of network centric warfare. 
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Figure 18. Snapshot of SA Map (left) and Inorganic SA Map (right), concerning 
Squad 1, for the same time of the simulation run. Fellow squad 
members are represented by triangles, other friendly targets are 

represented by inverted triangles, enemies are represented by (red) 
squares and neutral targets are represented by (blue) inverted 
squares. Unknown contacts are represented by white rectangles. 


e Fuel is one more innovative property of MANA. It does not 
necessarily refer to fuel, food or other supplies, but it can be used 
to record interactions with other agents or to assist in shifting into 
certain trigger states. The modeler can define several parameters 
such as refuel range, probability to refuel enemy/ friend/ neutral and 
refuel rate. This feature is used extensively in our model and will be 
explained in detail later. 
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C. MODEL IMPLEMENTATION 
1. Models in MANA 


As described in previous chapters, the Aegean Sea is an extremely 
complex environment because it consists of areas either of “open sea,” where a 
ship can sail for about 50 nautical miles without the “coverage” of nearby islands, 
or areas where “many islands” are very close to one another. These rare 
geographical conditions compose a challenging environment operationally. The 
effectiveness of UAVs is studied separately in those two environments creating 
two different models, as described in the following sections. 


a. “Open Sea” Model 


The “Open Sea” model represents a typical area of the Northern 
and Southern Aegean Sea. In this model (Figure 19), the Red Force consists of 
four surface vessels. These vessels attempt to transport illegal immigrants to the 
small island on the Southwest. To succeed in their mission they have to sail the 
open sea without being detected by the Blue Surveillance System. To increase 
the possibility of going undetected, they keep close to nearby merchant vessels 
that have approximately the same course with them. They have no 


communication capabilities, but it is assumed that they have the same tactic. 
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fa Terran Type New Coast added 
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Figure 19. “Open Sea” model implemented in MANA. 


The Blue Force, which has the responsibility to monitor the AOI, 
consists of two ground radars, located on the Northwest and Southeast islands 
respectively, as well as two Blue surface ships which patrol in two different 
sectors. In variations of this model, we study the effect of one or two UAVs on the 


performance of the surveillance system. 


In the area, there are a large number of merchant vessels (yellow 
ships) that sail along specific transit lanes either northbound, or southbound. In 
the Northeast corner are the Dardanelles Straights, which link the East 
Mediterranean with the Black Sea, and are usually characterized by high traffic. 
Additionally, there are randomly scattered fishing vessels (green ships), 


something very common in the area. 


b. “Many Islands” Model 


The “Many Islands” model represents a typical area of the Central 
and Southeast Aegean Sea. In this model (Figure 20), the Red Force consists of 
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four surface vessels which, again, attempt to transport illegal immigrants beyond 
the islands on the West. To succeed in their mission they have to sail westbound 
without being detected by the Blue Surveillance System. To increase the 
possibility of going undetected, they do not sail directly to the final destination; 
instead they sail close to the shoreline during their route. They have no 


communication between them, but it is assumed that they have the same tactic. 
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Anim ath 
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x= 60,0001 n. miles, y = 14.1692 n. miles Real time = 0.0 seconds Real time = 0 hour 0 min 0 sec Total SA Contacts: 0 


Figure 20. “Many Islands” model implemented in MANA. 


The Blue Force, which is responsible for monitoring the AOI, 
consists of two ground radars located on the West and Southeast islands, 
respectively, as well as two Blue surface ships that patrol in two different sectors. 
In variations of this model, we study the effect of one or two UAVs on the 
performance of the surveillance system. 


In the area, there are a large number of merchant vessels (yellow 


ships) that sail in various routes. This represents the movement of merchant 
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vessels, cruise ships or coastal shipping. Additionally, there are randomly 
scattered fishing vessels (green ships), something very common in the area. 


2. Assumptions 


Every model is subject to assumptions which limit it in specific 
circumstances. The models that are demonstrated in this study have the 
following assumptions: 


° No intelligence from other resources is taken into account. The 
Blue and Red forces are based only on the information taken from 
their own sensors. 


° The Red ships cannot communicate with each other to exchange 
information about the adversaries. 


e The size of the Red surface vessels is expected to be medium, 
such as fast patrol boats or fast motor boats. This factor affects the 
detection range, as explained in Chapter II. 


e The Red ships do not start sailing towards their goal unless they 
meet a merchant vessel that is sailing to the same area (“Open 
Sea” model). 

e The Blue UAVs and Blue ships get close to the enemy and remain 


there for a period of time only once. Operationally, it is reasonable 
for Blue sensors to gather information about a specific target one 
time. When a target has been lost, but we expect it to be in a 
specific area, we are satisfied if only one of our sensors locates it. 


e Blue ships and UAVs spend time close to enemy targets only to 
collect information (photos, electronic emissions, etc.). This is not 
necessary for merchant and fishing vessels because once it is 
known that the contact is neutral, the Blue units immediately move 
to the next unidentified contact or, if there is none, along their patrol 
route. 


3. Measures of Effectiveness (MOE) 


It is desirable for the Blue Force to monitor the AOI consistently. That 
means that we want them to detect and identify as many targets as they can but 
also to keep contact with the targets as much time as possible. Detecting and 
then losing a target gives only a piece of information on the target’s existence 


and nothing about its current location. For this reason, it is considered that a 
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more robust MOE to evaluate the system’s performance is the “Proportion of time 
Red Ships are positively identified” and “Proportion of time Merchant Vessels are 
positively identified.” We expect the two MOEs to be highly correlated, but we 
also want to investigate whether the surveillance system performs differently with 


respect to the type of contact. 


4. Agents Description 


As mentioned earlier, a squad is a group of agents that have the same 
characteristics. Every agent has a unique allegiance that depicts the agent’s side 
during the model run. It can be on the Blue side (Allegiance 1), on the Red side 
(Allegiance 2) or neutral (Allegiance 0). MANA offers additional features, such 
“Threat Level” and “Class,” so that differentiations among the agents can be 
expressed. All the agents used in the “Open Sea” model, along with their 
allegiance, threat level and class, are shown in Table 10. The same agents with 


similar characteristics are also used in the “Many Islands” model. 
































Table 10. Agents used in the “Open Sea” and “Many Islands” models 
Agent’s Name Agent’s Agent’s Threat Agent’s 
Allegiance Level Class 
Blue UAV 1 3 7 
Blue Ship 1 3 1 
Blue Ground Radar 1 1 3 3 
Blue Ground Radar 2 1 3 4 
Blue Operational Centre 1 2 0 
Red Ship 2 3 0 
Merchant Vessel 0 3 0 
Fishing Vessel 0 1 0 
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a. Blue Ships 


In every model there are two surface Blue ships. At the beginning 


of each scenario, the Blue ships patrol along a specific route covering different 


sectors. Together, along with the other Blue sensors described in this section, 


they cover all the AOI. The patrol route is focused near the coastline, as well as 


inside the territorial waters, to detect potential hidden targets. They exchange 


information with the Operational Centre to have improved SA. Initially, a Blue 


ship has a higher propensity to move towards unknown contacts in order to 


identify them (Figure 21). If it meets an enemy (Red ship), it sails towards it 


(“Enemy Contact 3” state) and stays close to it for more than 15 minutes (1000 





time steps). 
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Figure 21. Personalities tab of the Blue ship agent for the default state. 


This time is provided to collect all the information needed (photos, 


electronic emissions, etc.). Nothing is capable of interrupting this mission. After 


this time has passed, the Blue ship shifts to the “Spare 1” state where it shoots at 


59 


the enemy (with zero probability of success). This is only to help the Red ship to 
shift to the “Shot At” state, so that the Blue unit breaks lock of it. Once the Blue 
ship stops monitoring the enemy target, it sails towards other unidentified 
contacts or, if there is none, along its patrol route. To avoid engaging on the 
same target with other friendly forces, the Squad Friendly Awareness has been 
set to -100. Finally, the Blue ship is equipped with a detecting sensor. In this 
study the Advanced Sensor Model is used. The detection range has been divided 
into five zones, and each one has different detection probability. It is more 
accurate to consider that a target closer to the detecting sensor has a higher 
probability of being detected than a target close to the maximum range. The 
same method is applied for the Classification Probabilities (Figure 22). Finally, 
when a Blue ship identifies a target, it “refuels” it with a rate of 1 fuel unit per time 
step. At the end of each simulation run, the remaining “fuel” is used to calculate 


the time a target was monitored. 
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Figure 22. Sensors tab of the Blue ship agent. 
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b. Blue UAV 


Every model is studied having one or two UAVs available. The Blue 
UAVs have similar behavior to the Blue ships. If one UAV is available, it starts its 
patrol from the Northwest area (Figure 23), where the first Blue ship is located. 
The patrol route is focused near the coastline, as well as inside territorial and 
international waters, to detect potential hidden targets. If a second UAV is 
available, it starts from the Southeast area in the “Open Sea” model, where the 
second Blue ship is located (in the case of the “Many Islands” model, it starts 
from the Southwest area). In that case, the two UAVs cover different sectors. 
Together, along with the other Blue sensors described in this section, they cover 
all the AOI. They pass information to the Operational Centre and receive 


information from the nearest ground radar. 
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Figure 23. Map Tab of the Blue UAV agent in the “Many Island” model with one 
UAV available. 
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Initially, the Blue UAV has a higher propensity to move towards 
unknown contacts in order to identify them. If it meets an enemy (Red ship), it 
sails towards it (“Enemy Contact 3” state) and stays close to it for more than 15 
minutes (1000 time steps). This time is provided to collect all the information 
needed (photos, electronic emissions, etc.). Nothing is capable of interrupting 
this mission. After this time is passed, the Blue UAV shifts to the “Spare 1” state, 
where it shoots at the enemy (with zero probability of success). This is only to 
help the Red ship to shift to the “Shot At” state, so that the Blue unit breaks lock 
of it. Once the Blue UAV stops monitoring the enemy target, it sails towards other 
unidentified contacts or, if there is none, along its patrol route. To avoid engaging 
on the same target with other friendly forces, the Squad Friendly Awareness has 
been set to -100. Finally, the Blue UAV is equipped with a detecting sensor. In 
this study the Advanced Sensor Model is used. The detection range has been 
divided into five zones, and each one has different detection probability. The 
same method is applied for the Classification Probabilities. Finally, when a Blue 
UAV identifies a target, it “refuels” it with a rate of one fuel unit per time step. At 
the end of each simulation run, the remaining “fuel” is used to calculate the time 


a target was monitored. 


Cc. Blue Ground Radars 


The Blue ground radars have a limited FOV of 270 degrees. This is 
reasonable because none of the current ground radars in the Aegean Sea has a 
peripheral “visible” sector. They send all the information about the targets 
detected to the Blue Operational Centre. They also provide information to the 
available UAVs. The Advanced Mode for the sensors is used. The maximum 
detection and classification ranges are subdivided into five zones, and each one 
has different probability of detection/classification. This captures the intuitive 
opinion that if a contact is closer to the radar, it has more chances to be 
detected. While the radar has a contact under positive identification, it “refuels” it, 
and this helps us to keep track of the time the contact is classified. 
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d. Blue Operational Centre 


The role of the Blue Operational Centre is to gather the information 
from the Blue sensors and send them to the Blue ship (Squad 1) and Blue UAV 
(Squad 7) (Figure 24). The Operational Centre does not have detection 
capabilities itself. In reality, it can be located either away from the AOI or onboard 
a Command ship. 
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Figure 24. Inter squad SA table of the Blue Operational Centre. 


e. Red Ships 


In every model there are four Red ships. Depending on the model, 
they have different tactics. In the “Open Sea” model, they approach the closest 
merchant vessel and stay close to it for more than 2.5 hours (10,000 time steps) 
while transiting the open sea. During this tactic, they increase their concealment 
in order to avoid detection by the Blue surveillance system. In the “Many Islands” 
model, the Red ships prefer to stay close to the shoreline in order to reach their 


final destination. Again, using this tactic, they increase their concealment. 
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When the Red ships have been identified, they shift to the “Shot At” 
state and become invisible allowing the Blue UAV/ships to move to the next 
unidentified contact. They stay in this state for 30 minutes (1800 time steps), 
where they are considered as identified. Operationally, this is reasonable 
because in such a short time it is extremely difficult to mislead the surveillance 
system and avoid the detection. While the vessels are under positive 
identification, they consume “fuel,” and this helps us to keep track of the time the 
contact is classified. They do not have communication and weapon capabilities. 
They are equipped with a detection sensor operating in Simple Mode (cookie- 
cutter), as depicted in Figure 25. 
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Figure 25. Sensors tab of the Red ship agent. 


f. Merchant Vessels 


Merchant vessels are randomly distributed in the AOI. Their routes 
are consistent with the real merchant shipping lanes in the Aegean Sea. They do 


not have communication and weapon capabilities. Once they are classified by 
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the Blue sensors, they are assumed as such for the next hour. For this time 
period, they shift to the “Refuel by Friend” state and increase their concealment 
to 100 percent. That makes them invisible allowing the Blue Ship/UAV to move to 
the next unidentified contact. When the one-hour period expires, they have to be 
revisited in order to be re-identified. Operationally this is reasonable because 
merchant vessels usually keep constant speed and heading, and this fact 
facilitates the users of the surveillance sensors to keep track of them. While the 
vessels are under positive identification, they consume “fuel,” and this helps us to 


keep track of the time the contact is classified. 


g. Fishing Vessels 


Fishing vessels are randomly distributed in the AOI. They remain 
constant throughout the scenarios. They do not have communication and 
weapon capabilities. Once they are classified by the Blue sensors, they are 
assumed as such for the next two hours. For this time period, they shift to the 
“Refuel by Friend” state and increase their concealment to 100 percent. That 
makes them invisible allowing the Blue Ship/UAV move to the next unidentified 
contact. When the two-hour period expires, they have to be revisited in order to 
be re-identified. Operationally this is reasonable because fishing vessels remain 
approximately at the same position for long periods of time without trying to 
mislead the surveillance system. While the vessels are under positive 
identification, they consume “fuel,” and this helps us to keep track of the time the 
contact is classified. 
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IV. DESIGN OF EXPERIMENT 


A. EXPERIMENTAL SETUP AND FACTORS 


This section provides a brief description of the experimental setup that is 
followed in this study. Additionally, the factors that are varied in the Design of 
Experiment (DOE), along with their ranges, are presented. 


1. Experimental Setup 


In this study, two different scenarios are studied. The first is the “Open 
Sea” scenario in which the enemies have to transit long distances without the 
coverage of nearby islands to reach their final destination. This scenario 
represents the situation in the Northern and Southern Aegean Sea. The second 
scenario, which is found in the Central and Southeast Aegean Sea, is called 
“Many Islands” because it is characterized by a large number of islands being 
close to each other. The Red ships try to take advantage of this situation. The 
two scenarios are studied separately, and the conclusions of both will shed light 
on a part or the whole AOI. 


The two scenarios explore different combinations of Blue sensors and Red 
tactics. Concerning the Blue forces, the current surveillance system represented 
by two ground radars and two surface ships will be contrasted with some 
proposed solutions. The first solution involves one UAV in addition to the current 
system. In the second solution, two UAVs support the operations of the current 
surveillance system. The last solution is more challenging since the use of two 
UAVs along with two ground radars is being studied without the support of 
surface ships. 


From the enemy’s perspective, two tactics are studied. When the Red 
ships aim to reach a destination without having the option to sail close to nearby 
islands, they approach transiting merchant vessels and keep close to them, as 
much as they can. This tactic is captured into the “Open Sea” scenario. In 


contrast, when the environment is characterized by many islands in close 
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distances, the Red ships try to take advantage of the islands. Sailing near to the 
shoreline, they try to avoid being detected by the adversary’s sensors. This 
situation is captured in the “Many Islands” scenario. A combination, or further 
investigation, of the enemy’s tactics is not part of this thesis. The experimental 


setup just described is shown in Table 11. 
































Table 11. Experiment setup. 
“Open Sea” Scenario “Many Islands” Scenario 
no UAV no UAV 
Blue Sensors 1 UAV 1 UAV 
Combinations 2 UAVs 2 UAVs 
no ships/ 2UAVs no ships/ 2UAVs 
Approach merchant 
Red Tactics Approach shoreline 
vessels 
2: Design Factors 


In every experiment, it is impossible to account for every factor that might 
affect the outcome. Investigating too many factors would make the analysis 
extremely difficult, whereas having too few factors might result in inaccurate 
conclusions. This study varies 21 factors (Table 12) that seem to have the most 
important impact on the performance of the surveillance system. The factors 
were chosen based on opinions of subject matter experts, as well as the author's 
experience. Although in simulation experiments all the factors are manipulated, in 
reality this is not the case. For this reason, the factors are distinguished into 


controllable and uncontrollable (Sanchez & Wan, 2012). 
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Table 12. 


Independent factors varied in the DOE. 








































































































Platform Variable Type Unit Range 
1. Speed Continuous 
(sampled in knots 70-150 
discrete values) 
2. Max Detection Range Csnintious weeks 5000- 
30000 
3. Avg. Time Between 
Detections (Max range) Continuous seconds 5-120 
UAV 
4. Probability of Classification 
at Min Range Continuous 0.7 1 
e COMUNE Hon alone, Continuous seconds 30-120 
Bs: Compunication Acaracy Continuous percentage 50-100 
7. Speed Continuous 
(sampled in knots 15-25 
discrete values) 
8. Blue Mar Dereham lange Continuous meters S000 
: 25000 
9 Bales Avg. Time Between 
: c Continuous seconds 8- 240 
Detections (Max range) 
10. Probability of Classification ; 
at Min Range Continuous 0.7 1 
11. Max Detection Range ; 10000- 
Continuous meters 50000 
Blue Avg. Time Between 
12.| Ground g. Continuous seconds 5-120 
Detections (Max range) 
Radars 7 as 
13. Probability of Classification Gonunuode 0.7 1 
at Min Range ‘ 
14.| Opera- Communication Latency Continuous seconds 30-120 
tional — 
15. | Gentre | Communication Accuracy Continuous percentage 50-100 
16. Stealth Continuous percentage 30-90 
17. Detection Range Cantiauoue si atigs 5000- 
Red Ships 25000 
18. Speed Continuous 
(sampled in knots 15-35 
discrete values) 
19.| Fishing # of fishing vessels Disetete : 5-50 
Vessels 
20. # of merchant vessels Discrete “ 4- 30 
241. | Merchant speed Continuous 
vessels (sampled in knots 8-20 











discrete values) 
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The controllable factors are those that are varied at will by the 
experimenter. All the variables that deal with the Blue sensors (Blue ships, UAVs, 
ground radars and Operational Centre) fall in this category. These variables 
describe the detection capabilities of the Blue sensors and possibly have an 
effect, major or minor, to the response variable. These factors include the 
detection and classification ranges, the probabilities of detection, speed and 


communication capabilities. 


Uncontrollable or noise factors are those that cannot be manipulated by 
the decision maker in reality. In this experiment, the number of merchant or 
fishing vessels, their speed and the configurations of the Red ships fall into this 
category. They may or may not have a significant effect on the outcome, but by 
varying within valid ranges, they introduce a natural variability in the models. In 


that way more accurate conclusions are drawn. 


In addition to the 21 independent variables that we explicitly vary in the 
DOE, there are an additional 52 variables that depend on one of the 
aforementioned factors. These variables describe the sensors’ capabilities, and 
this technique is extremely useful when the Advanced Detection Mode is used 
(see Chapter III). 


The Maximum Classification Range is set to 0.8* Maximum Detection 
Range for all the agents. Moreover, all the Blue units use the Advanced 
Detection Mode where a range table can be set up to define how the sensing 
capability falls off with increased distance. Therefore, their detecting ranges are 
divided into five homocentric cycles, each one with a different chance to detect 
(or classify) a target. The “upper” range value for each of the four inner cycles is 
calculated by multiplying the maximum detection (or classification) range by 1/5, 
2/5, 3/5 and 4/5, respectively. For example, if the maximum detection range is 
25,000 meters, Zone 1 will be up to 5,000 meters, Zone 2 from 5,000 to 10,000 


meters, etc. 
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For the detection, those “zones” are associated with Average Times 
between Detections. As indicated in the previous paragraph, these times are 
calculated by multiplying the Average Times between Detections for the 


maximum detection (or classification) range by 1/5, 2/5, 3/5 and 4/5, respectively. 


For the classification, the five ranges are assigned with probabilities of 
classification. In this case, taking as a reference the first zone, which has the 
maximum probability of classification, we calculate the rest by multiplying 
successively by 0.9, 0.8, 0.7 and 0.6. In that way, at the verge of the 
classification range we obtain 60 percent of the nominal maximum probability of 


Classification. All the dependent variables are depicted in Table 13. 
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Table 13. 


Dependent variables in the DOE 








































































































































































































































































































Dependent Variable in BeReriea Osan Multiplying by 
1. Maximum Classification Range Maximum Detection 08 
(Zone 5) Range (Zone 5) 
2. 1st detection range (Zone 1) 0.2 
3. 2nd detection range (Zone 2) Maximum Detection 0.4 
| 4. | 3rd detection range (Zone 3) Range (Zone 5) 0.6 
5. 4th detection range (Zone 4) 0.8 
6. Time Between Detections (Zone 1) Tine Between 0.2 
7. Time Between Detections (Zone 2) Detections IN 0.4 
8. Time Between Detections (Zone 3) Maximum Range 0.6 
9. UAV | Time Between Detections (Zone 4) (Zone 5) 0.8 
10. 1st classification range (Zone 1) RA axaRiunn 0.2 
11. 2nd classification range (Zone 2) Classification Range 0.4 
12. 3rd classification range (Zone 3) (Zone 5)* 0.6 
13. Ath classification range (Zone 4) ("itis also dependent) 0.8 
14. Probability of Classification (Zone 2) 0.9 
15. Probability of Classification (Zone 3) Probability of 0.8 
a ae Classification at Min 
16. Probability of Classification (Zone 4) Range (Zone 1) 0.7 
17. Probability of Classification (Zone 5) 0.6 
18. Maximum Classification Range Maximum Detection 0.8 
(Zone 5) Range (Zone 5) 
19. 1st detection range (Zone 1) 0.2 
20. 2nd detection range (Zone 2) Maximum Detection 0.4 
21. 3rd detection range (Zone 3) Range (Zone 5) 0.6 
22. 4th detection range (Zone 4) 0.8 
23. Time Between Detections (Zone 1) Tine: Between 0.2 
24. Time Between Detections (Zone 2) Detections IN 0.4 
25. Time Between Detections (Zone 3) Maximum Range 0.6 
26. an Time Between Detections (Zone 4) (Zone 5) 0.8 
27. 1st classification range (Zone 1) Maximum 0.2 
28. 2nd classification range (Zone 2) Classification Range 0.4 
29. 3rd classification range (Zone 3) (Zone 5)* 0.6 
30. 4th classification range (Zone 4) ("itis also dependent) 0.8 
31. Probability of Classification (Zone 2) 0.9 
32. Probability of Classification (Zone 3) Probability of 0.8 
33. Probability of Classification (Zone 4) | Classification at Min 0.7 
34. Probability of Classification (Zone Range (Zone 1) ae 
5 : 
35. enn Classification Range Maximum Detection 08 
Ground (Zone 5) Range (Zone 5) 
36. | Radars 1st detection range (Zone 1) Maximum Detection 0.2 
37) 2nd detection range (Zone 2) Range (Zone 5) 0.4 
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38. 3rd detection range (Zone 3) 0.6 
39. 4th detection range (Zone 4) 0.8 
40. Time Between Detections (Zone 1) Tine Between 0.2 
41. Time Between Detections (Zone 2) Detections IN 0.4 
42. Time Between Detections (Zone 3) Maximum Range 0.6 
43. Time Between Detections (Zone 4) (Zone 5) 0.8 
44. 1st classification range (Zone 1) Maxinidin 0.2 
45. 2nd classification range (Zone 2) Classification Range 0.4 
46. 3rd classification range (Zone 3) __ (Zone 5)" 0.6 
47. 4th classification range (Zone 4) (* itis also dependent) 0.8 
48. Probability of Classification (Zone 2) 7 0.9 
49. Probability of Classification (Zone 3) Probability of 0.8 
a op Classification at Min 
50. Probability of Classification (Zone 4) Range (Zone 1) 0.7 
51. Probability of Classification (Zone 5) 0.6 
92. oa Maximum Classification Range Merandny Beterion 0.8 
Ships Range 





B. NEARLY ORTHOGONAL LATIN HYPERQUBE DESIGNS 


Latin Hypercubes (LH) have been increasingly popular in the designs of 
computer experiments because they provide a flexible way of constructing 
efficient designs to explore quantitative factors. They have “some of the space- 
filling properties of factorial designs with fine grids but require orders of 
magnitude less sampling” (Sanchez & Wan, 2011). If we had a small number of 
factors, we could create a full factorial design to study all possible combinations 
of the factor levels. However, this is impossible when we deal with a large 
variables because the time needed would increase 


number of input 


geometrically. 


Nearly Orthogonal Latin Hypercubes (NOLH) are “LH which have good 
space-filling and orthogonality properties for a small or moderate number of 
factors. These designs are not square, but the number of design points is 
radically fewer than the numbers for the gridded designs” (Sanchez & Wan, 
2011). 


Cioppa and Lucas (2007, pp. 45-55) presented NOLH with good space- 
filling properties for up to 22 factors in as few as 129 input combinations, and 


later up to 29 factors in 257 design points. An Excel spreadsheet, developed 
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from the Simulation Experiments and Efficient Design (SEED) Center for Data 
Farming at the Naval Postgraduate School in Monterey, is used to create those 
specific combinations that the NOLH design demands, as shown in Figure 26. 








Figure 26. STapehol of the spreadsheet used for the DOE. 


There are three major benefits of NOLH in this study: 


° Only 257 design points are necessary in this DOE. Instead, if we 
wanted to explore the 23 factors as 2-level, and not as continuous, 
we would need more than eight million design points. 


e It is a good space-filling design because “the design points are 
scattered throughout the experimental region with minimal 
unsampled regions” (Cioppa & Lucas, 2007). Figure 27 is a 
scatterplot matrix that depicts the projections of the full design onto 
each pair of factors. It can be easily seen that this property holds. 
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Figure 27. Scatterplot Matrix for all the factors in the DOE. 





e Near orthogonality ensures that all correlations between the factors 
are very small, between the interval (-0.03, 0.03). Figure 28 
confirms this property for the first ten factors because the maximum 
absolute pairwise correlation is 0.0096. 
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4 Correlations 
UAV_SpeedUAV_MaxDeRange UAV _AupTimeBweenDelUAV_PrClass UAV_CommsLatencyUAV_CommsAccuracy BlueShips_ Speed BlueShips MaDetRange BlueShips_AvgTimeBweenDetBlueShips PrClass BlueShips Ca 
UAV Speed 1.0000 ‘ 
UAV MarDetRange 4.0000 
UAV AvoTimeBweenDet 1.0000 


Hl ny 1.0000 
BlueShips Speed { 10000 
BlueShips_MaxDetRange 1.0000 
BlueShips_AgTimeBweenDet 10000 
BlueShips_PrClass ‘ 0000 
BlueShips_CommsLatency 
BlueShips_CommsAccuracy 
Radat_MarDeRange 
Radar_AugTimeBweenDet 
Radar_ProoClass 
OpCentre_CommsLatency 
Ap ny 










Red Stealth 

Red_DetRange 
Red_Speed 
NumberFish 


Figure 28. Snapshot of correlation matrix for the first ten factors of the DOE. 


C. EXPERIMENT EXECUTION 


Our simulation experiment was run on a high-performance cluster with 44 
processors, called “Reaper.” The software packages xStudy, OldMcData and 
Condor were used in this study to generate the data files. XStudy, written by 
SEED Center Research Associate Steve Upton, is a graphical user interface for 
generating a study design file in .xml format. Among other information, it 
identifies the model used, the number of replications, initial random seeds and 
what variables will be used for variation. It is also used to map each factor of the 
DOE to a specific element of the MANA scenario file. 


OldMcData (OMD) is a software application designed to do data farming 
runs, from running large simulation experiments on a distributed computer cluster 
to multiple replications of a single excursion on a single machine. For runs on a 
distributed computing cluster, OMD uses Condor, an open-source distributed 


computing environment, to handle the scheduling and managing of the active 


76 


jobs. All the software mentioned in the previous paragraphs is available through 
the NPS SEED Centre’s website (http://narvest.nps.edu/). 


V7 


THIS PAGE INTENTIONALLY LEFT BLANK 


78 


V. RESULTS AND DATA ANALYSIS 


A. MEASURES OF EFFECTIVENESS AND DATA PREPARATION 


This section describes the calculation made to obtain the two MOEs and 
the reasons that led us to manipulate the data taken for the 1-UAV case. 
Additionally, this section contains our study about the necessary assumptions 
that have to be met to conduct parametric statistical methods. 


1. Measures of Effectiveness 


As illustrated in Chapter Ill, two Measures of Effectiveness (MOE) are 
used. The first is “The proportion of time the enemy ships are positively 
identified.” This MOE, referred to as MOE 1, estimates how the surveillance 
system of the Blue Force performs. This measure is indirectly calculated using 
the amount of “Fuel” of each Red Agent. At the beginning of a run, each one of 
the four Red ships is equipped with a certain amount of fuel that is set to remain 
constant any time, unless it is identified by the Blue Force. For the time period 
the targets are monitored, they consume one unit of fuel per time step. At the end 
of each simulation run, the fuel that all the Red Ships have consumed is summed 
up and divided by the number of Red Ships and the Total Run Time. In both the 
scenarios, there are four Red Ships that try to reach their goal without being 
detected. Finally, the result of MOE 1 is a number expressed as a percentage. 
Formula (1) describes the calculation of MOE 1: 


Sum of Fuel Consumed by Red Ships 


4* Total Run Time 


The second MOE, referred to as MOE 2, is “The proportion of time the 
merchant vessels are positively identified.” Although we are mostly interested in 
the surveillance performance over the Red Ships, this metric is used as an 
element of force validation to the simulation model. It is expected that the second 


MOE will be higher in all cases because Merchant Vessels do not use any tactics 
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to avoid detection from the Blue Force. Additionally, it will be interesting to see to 
what extent this metric is affected by the various combinations of the Blue 
Sensors. 


The same procedure for MOE 1 is used to calculate MOE 2. Like Red 
Ships, Merchant Vessels consume “fuel” for the time period they are monitored 
by the surveillance system. The overall amount of fuel that the Merchant Vessels 
consumed is divided by the aggregated time they were in the simulation run. 


Formula (2) describes the calculation of MOE 2: 


Sum of Fuel Consumed byMerchant Vessels 
k * Total Run Time 


MOE2= 





4) 
where k is the number of Merchant Vessels in the model. 


The “Open Sea” model involves 16 Merchant Vessels, and the “Many 


Islands” model involves 10. Again, the result yields a percentage. 


2. Data Preparation 


Concerning the case when only one UAV is deployed, the preliminary runs 
showed that the area from where it is launched affects the performance of the 
UAV, with respect to the MOEs, and consequently, of the whole surveillance 
system. When two UAVs operate, the AOI is equally divided in two sections and 
each UAV is assigned in a designated section. However, when only one is 
available, it covers all the AOI by itself. According to the attributes that have been 
given to the UAV, it patrols along a particular route until it locates an unknown 
contact. Then it heads towards it to identify it. As it is easily understood, the more 
contacts there are close to the launch site, the more the UAV contributes to the 
surveillance system. That is, if the launch site is close to areas with traffic, the 
UAV will have more opportunities to locate the targets. Additionally, the targets 
will be identified earlier. The difference in the MOE 1 with respect to the location 
of a single UAV launched for the “Open Sea” Model is shown in Figure 29. When 


the UAV started its deployment from the Northwest, the Surveillance System 
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detected the Red Ships on average 60.7% of the time (SD=O0.057). In contrast, 
when the UAV started its patrol from Southeast, it scored 56.9% (SD=0.056). An 
independent sample t-test with a two tailed alpha level of .05 indicated that 
scores were significantly higher for the first case than the second (t(998)= -9.381, 
p<0.0001). The same issue also occurred in the “Many Islands” model 
(t(998)= -5.244, p<0.001). More details are provided in Appendix A. 


4 \~ Distributions Case=OpenSea-1UAV (North) 
4 ~) %RedClassified 





4 Quantiles 4 ~) Summary Statistics 
100.0% maximum 0.73841 Mean 0.6032692 
99.5% 0.72322 Std Dev 0.0575825 
97.5% 0.7078 Std Err Mean 0.0025752 
90.0% 0.67365 Upper 95% Mean 0.6083287 
75.0% quartile 0.64434 Lower 95% Mean 0.5982097 
50.0% median 0.60535 N 500 
25.0% quartile 0.57134 

045 05 055 06 065 0.7 0.75 10.0% 0.52302 
2.5% 0.46893 
0.5% 0.43495 


0.0% minimum 0.42922 
4 \~ Distributions Case=OpenSea-1UAV(South) 
4\~ %RedClassified 





4 Quantiles 4 \~) Summary Statistics 
100.0% maximum 0.70788 Mean 0.5695073 
99.5% 0.69359 Std Dev 0.0562148 
97.5% 0.6767 Std Err Mean 0.002514 
90.0% 0.64655 Upper 95% Mean 0.5744466 
75.0% quartile 0.60342 Lower 95% Mean 0.5645679 
50.0% median 0.56954 N 500 
25.0% quartile 0.53945 

0.35 0.4 0.45 0.5 0.55 0.6 0.65 0.7 10.0% 0.49994 
2.5% 0.4448 
0.5% 0.39202 


0.0% minimum 0.36637 


Figure 29. Summary statistics for when a single UAV is launched from Northwest 
or Southeast in the “Open Sea” model. 


In order to mitigate the dissimilarities in geography and _ target 
arrangements, two different scenarios were run every time one UAV is needed. 
In the first scenario, the UAV is launched from a location in the Northwest, 
whereas in the second, the UAV started its deployment from the Southeast 
(“Open Sea” model) or the Southwest (“Many Islands” model), respectively. 
When all the data had been collected, we calculated the averages of the MOEs 
of the two 1-UAV scenarios. In that way, it can be considered that the UAV was 


launched from a location in the middle of the true launch sites. 
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3. 


Assumptions for Statistical Analysis Method 


Our decision whether to use parametric or non-parametric method for the 


analysis is based on whether the assumptions of independence, equality of 


variance, and normality are met. Below we study those assumptions for the 


“Open Sea” model. 


Independence of replications: The observations within each 
treatment group must be independent of one another. Our 
experiment was completely randomized because we derived our 
data through a stochastic model. MANA uses a random number 
generator to randomize the starting positions of agents within their 
defined home boxes as well as the probabilities of detection. 
Consequently, scenarios can have considerably different outcomes 
each time they are run. Therefore, the assumption of independence 
holds. 


Equality of Variance: The treatments must have approximately 
equal variances. The side-by-side boxplots for the four cases we 
are interested in for the “Open Sea” Model are shown in Figure 30. 
It can be seen that the variance is approximately the same across 
the different cases. Additionally, because we have a large sample 
size, we can assume that the equal variance assumption holds. 
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Figure 30. Side-by-side boxplots for the four scenarios in the “Open Sea” model. 


Normal Population: The data of each treatment must come from 
populations that are Normal or close to Normal. The histograms of 
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Residuals for the four cases are shown in Figure 31. Although the 
histogram is skewed on the left in the case where there are no 
UAVs, it can be considered that all the cases adequately meet the 
Near Normality Assumption. 


Case vs. Residual %RedClassified 
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Figure 31. Histogram of the residuals across the four cases in the “Open Sea” 
model. 


The same assumptions also hold for MOE 2 as well as for both MOEs in 
the "Many Islands" model, as presented in Appendix B. In addition, Appendix C 
contains the study for the assumptions for both the MOEs for the “Many Islands” 
Model. Finally, a two-tailed alpha level of .05 was employed for all statistical 
tests. 


B. RESULTS OF INITIAL RUNS 


This section studies the results of 500 initial runs that were conducted for 

each one of the “Open Sea” and the “Many Islands” models. These runs used a 

particular configuration of the capabilities of the agents involved in the models. 
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The results from the simulation were used to compare the performance of the 
surveillance system in the different scenarios with respect to the MOEs defined 
earlier. Descriptive statistics of the initial runs are presented and hypotheses for 
the research questions are tested using appropriate statistical methods. Results 


from the "Open Sea" model are presented first. 


1. “Open Sea” Model 
a. Descriptive Statistics 


The distributions of the percentage of Red Ships that had been 
classified across the four different cases are shown in Figure 32. We notice that 
on average, the surveillance system yielded the highest numbers of detections 
when two UAVs operate (Mean= 0.668, SD=0.052). In contrast, the current 
surveillance system with no UAVs performed the worst (Mean= 0.419, 
SD=0.049). 








» Distributions Case=OpenSea-1UAV 4 ~ Distributions Case=OpenSea-noBlueShips 
4» %RedClassified 4» %RedClassified 
4 Quantiles 4» Summary Statistics 4 Quantiles 4» Summary Statistics 
100.0% maximum 0.69475 Mean 100.0% maximum 0.67831 Mean 05281356 
99.5% 0.68386 StdDev 99.5% 0.669 StdDev 0.0611286 
97.5% 0.66508 Std Er Mean 97.5% 0.63975 Std Err Mean 0,0027338 
90.0% 0.64158 Upper95% Mean 0.5901056 90.0% 0.59387 Upper95% Mean 05335067 
75.0% quartile 0.61566 Lower95%Mean 0.5826708 75.0% quartile 0.57602 Lower95%Mean 05227645 
50.0% median 0.58953 N 500 50.0% median 0.53586 N 500 
25.0% quartile 0.55957 25.0% quartile 0.48657 
045 05 055 06 065 07 10.0% 0.52804 10.0% 0.4446 

25% 0.50013 25% 0.39687 
0.5% 0.47386 05% 0.36103 


0.0% minimum 0.29138 
~ Distributions Case=OpenSea-noUAV 


0.0% = minimum 0.46258 


» Distributions Case=OpenSea-2UAVs 





4» %RedClassified 4 ~~ %RedClassified 
4 Quantiles 4 > Summary Statistics 4 Quantiles 4») Summary Statistics 
100.0% maximum 0.78229 Mean 100.0% maximum 0.57141 Mean 0.4192253 
99.5% 0.75896 StdDev 99.5% 0.5633 StdDev 0.0490573 
a 975% 0.53761 Std Err Mean 0.0021939 
97.5% 0.74862 Std Err Mean 
90.0% 0.72837 Upper95% Mean 0.6731706 90.0% 0.46969 Upper95% Mean 0.4235358 
75.0% quartile 0.7055 Lower95% Mean 0,6639941 75.0% quartile 0.44553 Lower95%Mean 0.4149149 
50.0% median 0.67983 N 500 50.0% median 0.42137 N 500 
25.0% quartile 0.6388 25.0% quartile 0.39304 
0 1 
035 045 055 065 07: 10.0% 0.59809 0.25 0.3 035 0.4 0.45 05 055 us {ae 
25% 0.55091 ae aoe 


0.5% 0.48526 
0.0% minimum 0.37417 


0.0% minimum 0.27507 





Figure 32. Distributions of MOE 1 for the 500 initial runs across the four cases in 
the “Open Sea” model. 


“The distributions of the percentage of Merchant Vessels that had 


been classified across the four different cases are shown in Figure 33. As 
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expected, on average the surveillance system yielded the highest numbers of 
detections when two UAVs operate (Mean= 0.791, SD=0.033). In contrast, the 
current surveillance system with no UAVs performed the worst (Mean= 0.477, 
SD=0.036). 


+ Distributions Case=OpenSea-1UAV + Distributions Case#OpenSea-noBlueShips 
4 > %MerchantClassified 4 > %MerchantClassified 


4 Quantiles 4 > Summary Statistics — 4 Quantiles 4 = Summary Statistics 
100.0% maximum 0.75781 Mean 0.6684755 . 100.0% maximum 0.72336 Mean 0.6378681 
073599 SidDev 0.0308623 99.5% 0.71938 StdDev 0.0348566 
0.72574 Sid En Mean 0.0013802 97.5% 0.70824 StdEn Mean 0.0015583 
0.70771 Upper95% Mean 0.6711872 90.0% 0.69632 Upper 95% Mean 0.6409307 
quartte 0.66919 Lower95% Mean 0.6657637 75.0% quartile 0.66234 Lower95% Mean 0.6348054 
median 0.66999 N $00 50.0% median 0.62798 N 500 
quartie §=0,6485 25.0% Quartie § 0.6147 
058 062 066 O07 074 0.62445 054 058 062 066 07 10.0% 0.59906 
0.60568 25% 0.58388 
0.59273 05% 0.57023 
minimum 0.59039 0.0% minimum 0.54257 


~ Distributions Case=OpenSea-2UAVs 4 ~ Distributions Case=OpenSea-noUAV 


4» %MerchantClassified 4 > %MerchantClassified 


—- 4 Quantiles 4 > Summary Statistics — 4 Quantiles 4» Summary Statistics 
. 7 +—_(}——+ 100.0% maximum 086296 Mean 0.7916686 . »”® 100.0% maximum 0.62979 Mean 0.4778035 


99.5% 086778 Sid Dev 0.033532 99.5% 056937 StdDev 0.0366816 
97.5% 085381 Sid En Mean 0,0014996 97.5% 055478 StdEn Mean 0.0016405 
90.0% 083212 Upper95% Mean 07946149 90.0% 0.52549 Upper95% Mean 0.4810266 
75.0% quartie 081606 Lower95%Mean 0.7887223 75.0% Quartile 050162 Lower95% Mean 0.4745905 
50.0% median 0.79341 N 500 50.0% median 047614 N 500 
25.0% quartile 0.77119 25.0% quartile 0.45282 
10.0% 0.74765 035 04 045 05 055 06 065 10.0% 0.43303 
25% 0.72005 25% 0.40483 
05% 0.69742 05% 0.39052 
0.0% minimum 0.64157 0.0% minimum 0.36465 
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Figure 33. Distributions of MOE 2 for the 500 initial runs across the four cases in 
the “Open Sea” model. 


The average percentage of time the targets are identified is shown 
in Figure 34. Concerning the Red Ships, besides the best and worst cases which 
were commented on earlier, the 1-UAV case (which concludes also two Blue 
Ships) performs slightly better than the case where only two UAVs operate 


without any surface support. 
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Figure 34. Average percentage of time the targets are detected across the four 
cases of the “Open Sea” model. (Standard Error ranges from 0.0013 
to 0.0027 and would not be easily seen on the bars). 


It is worth noting that the Merchant Vessels results are consistent 
with those of the Red Ships. Additionally, in every case the Red Ships succeed in 
being detected fewer times than the Merchant Vessels using specific tactics to 
avoid the adversary sensors. This provides an element of force validation to the 


simulation model. 


b. Hypotheses Testing 


One of the primary questions of this thesis is whether UAVs are 
able to mitigate the incapability of the current surveillance system to detect 
targets in complex maritime environments, given that the target's tactic is to 
approach transiting merchant vessels. In other words, does adding UAVs to a 
surveillance system that uses traditional units, such as ships and radars, help it 
become more efficient? The hypothesis tested is: 


° Ho: there is no difference in the surveillance system 
performance whether it uses UAVs or not, given that the 
target's tactic is to approach transiting merchant vessels 
(U1=HW2="3=N4 where the four populations refer to the four 
different cases). 
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e Ha: a surveillance system with UAVs performs better than 
without UAVs, given that the target’s tactic is to approach 
transiting merchant vessels (W1#W2#U34U4 where the four 
populations refer to the four different cases). 


To test the hypothesis above, Analysis of Variance (ANOVA) was 
used. This parametric method demands particular assumptions, such as 
independence of groups, the equal variance assumption and the Normal 
Population assumption (De Veaux et. al, 2009). In the previous section, we 
proved that all these assumptions were met for both the MOEs. 


The ANOVA (R*=0.756) showed that the null hypothesis was 
rejected: there was a significant difference between the four cases (F(3, 
1996)=2061.5, p < 0.001) (Figure 35). That is, the number of UAVs has a 
significant effect on the performance of the surveillance system. 


4 Oneway Anova 
4 Summary of Fit 


Rsquare 0.75601 
Adj Rsquare 0.755643 
Root Mean Square Error 0.051624 
Mean of Response 0.550583 


Observations (or Sum Wots) 2000 
4 Analysis of Variance 


Sum of 
Source DF Squares Mean Square FRatio Prob>F 
Case 3  16.482292 5.49410 2061557 <.0001* 
Error 1996 5.319387 0.00267 


C. Total 1999 21.801679 
Figure 35. ANOVA for the initial runs of the “Open Sea” model. 
To uncover which UAV cases were significantly different from each 
other, two sample t-tests were conducted in terms of the mean proportion of time 


Red Ships were positively identified. All the pairwise differences after conducting 


Student’s ttest are shown in Table 14. We notice that, statistically, every case 
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differs from each other. Furthermore, the largest “improvement” occurs when we 


go from the current system (no-UAV) to a solution with one or two UAVs. 


Table 14. Ordered pairwise differences report after Student’s “test (“Open 
Sea’). 
4 Ordered Differences Report 
Level -Level Difference StdErr Dif LowerCL UpperCL p.Value 


OpenSea-2UAVs  OpenSeanoUAY  0.2493570 0.0032650 0.242053 0.2657601 < 0001" 
OpenSea-1UAY OpenSea-noVAY --0.1671629 0.032650 0.160798 0.1735660 <0001* 
OpenSea-2UAVs  OpenSea-noBlueShips 0.1404468 0.0032650 01340436 0,1468499 <0001* 
OpenSea-noBlueShips OpenSeanoVAY 0.1089103 0.032650 0.102507 0.1163434 <0001* 
OpenSea-2UAVs OpenSea-1UAY 0,0821941 0,0032650 0.0757910 0.088973 <0001* 
OpenSea-1UAV  OpenSea-noBlueShips 0.068226 0,0032650 0.0518405 0.0648568 <0001* 





2. “Many islands” Model 
a. Descriptive Statistics 


The distributions of the percentage of Red Ships that had been 
classified across the four different cases are shown in Figure 36. We notice that 
on average the surveillance system yielded the highest numbers of detections 
when two UAVs operate (Mean= 0.775, SD=0.036). In contrast, the current 
surveillance system with no UAVs performed the worst (Mean= 0.480, 
SD=0.062). 
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Figure 36. Distributions for MOE 1 for the 500 initial runs across the four cases in 
the “Many Islands” model. 


The distributions of the percentage of Merchant Vessels that had 
been classified across the four different cases are shown in Figure 37. As 
expected, on average the surveillance system yielded the highest numbers of 
detections when two UAVs operate (Mean= 0.914, SD=0.028). In contrast, the 
current surveillance system with no UAVs performed the worst (Mean= 0.780, 
SD=0.078). 
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Figure 37. Distributions for MOE 2 for the 500 initial runs across the four cases in 
the “Many Islands” model. 
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The average percentage of time the targets were identified is 
shown in Figure 38. Concerning the Red Ships, there is no big difference 
whether Blue Ships operate or not, given that two UAVs are deployed. It is worth 
noticing that the results are consistent for the case of Merchant Vessels, too, 
because the surveillance system performs similarly as before. Additionally, in 
every case the Red Ships succeed in being detected fewer times than the 
Merchant Vessels using specific tactics to avoid the adversary sensors. This set 


of results provides an element of force validation to the simulation model. 
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Figure 38. Average percentage of time the targets are detected across the four 
cases of the “Many Islands” model. (Standard Error ranges from 
0.0012 to 0.0035 and would be too small to be easily seen on the 
bars). 


b. Hypotheses Testing 


One of the primary questions of this thesis is whether UAVs are 
able to mitigate the incapability of the current surveillance system to detect 
targets in complex maritime environments, given that the target's tactic is to take 
advantage of the nearby shorelines. In other words, whether adding UAVs in a 
surveillance system that uses traditional units such as ships and radars, it 


becomes more sufficient. The hypothesis tested is: 
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e Ho: there is no difference in the surveillance system 
performance whether it uses UAVs or not, given that the 
target’s tactic is to take advantage of the nearby shorelines 
(U1=H2="3=N4 where the four populations refer to the four 
different cases). 


° Ha: a surveillance system with UAVs performs better than 
without UAVs, given that the target's tactic is to take 
advantage of the nearby shorelines (U1#U2#U34"4 Where the 
four populations refer to the four different cases). 


To test the hypothesis above, the Analysis of Variance (ANOVA) 
was used. This parametric method demands particular assumptions, such as 
independence of groups, the equal variance assumption and the Normal 
Population assumption. The study provided in Appendix C proves that the 


assumptions are met. 


Results indicated the null hypothesis was rejected. The ANOVA 
(R= 0.861) showed that there is a significant difference between the four cases 
(F(3, 1996)=4137.9, p-value< 0.001) (Figure 39). That is, the number of UAVs 
has significant effect to the performance of the surveillance system. 


4 Oneway Anova 
4 Summary of Fit 


Rsquare 0.861483 
Adj Rsquare 0.861275 
Root Mean Square Error 0.047343 
Mean of Response 0.680873 


Observations (or Sum Wots) 2000 
4 Analysis of Variance 


Sum of 
Source DF Squares Mean Square FRatio Prob>F 
Case 3  27.823305 9.27444 4137.935  <.0001* 
Error 1996 4.473674 0.00224 


C. Total 1999 32.296979 


Figure 39. ANOVA for the initial runs of the “Many Islands” model. 
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To determine which UAV cases were significantly different from 
each other in terms of the mean proportion of time that Red Ships are positively 
identified, multiple comparisons methods were employed (two sample f tests). All 
the pairwise differences after conducting Student’s t-test are shown in Table 15. 
We notice that, statistically, every case differs from each other. Furthermore, the 
largest “improvement” occurs when we go from the current system (no-UAV) to a 
solution with one or even two UAVs. 


Table 15. Ordered pairwise differences report after Student’s -test (“Many 
Islands”). 
Ordered Differences Report 

Level Level Difference StdErr Of LowerCl Upper Cl p-Value 

Wanylslands-2UAVS  Manylslands-noUAV 2952650 0.0020042 02893829 03041271. <000t 

Wanyslands-noBlueShips Manvslands-noUAY 02743301 00029042 02684580 02802022 < 0004" 

Vanjlands (UAV anja AY 0232398 0000942 205477 O23R0619 <OUUt] a 
Wanwslands-2UAVs Manvislands-1UAV (0828662 0.0020942 0.0689031 00687373 <0001* Lm 

Wanyslands-noBlueShips Manyslands-tUAY 00449403 0,0029942 00360682 00478124 <004* 

Wanylslands-2UAVS  Manylslands-noBlueShips 0.0200248 0,0020942 0.0150627 0.0267969 <0001 


C. EXPLORATORY ANALYSIS 


The purpose of this analysis is not to predict the time that on average the 
targets will be identified, but rather to understand the significant factors that affect 
the surveillance system’s ability to detect and identify enemies. For this 
exploratory analysis, we focused on MOE 1 with one UAV, utilizing Least 
Squares Regression and Partition Trees analytic methods. These two methods 
were then compared. This comparison was helpful to recognize the significant 
factors that contribute most to the response variable, although we expect some 
similarities between the two methods in terms of their overall importance. 


Interpretation of results from this comparison will be covered in Chapter VI. 
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1. “Open Sea” Model 


The “Open Sea” model with one UAV (and specifically the northern one), 
which was created in MANA, used the appropriate DOE that was presented in 
Chapter V. The 129 design points (DP) replicated 250 times each resulting in 
32,250 observations. Instead of using the raw data, we used the mean values of 
the replicated DP to reduce the calculations and produce more visually efficient 
displays. 


a. Regression Analysis 


We fitted both linear and non-linear Least Square Regression 
Models. The advantage of the former is that they can often approximate complex 
processes adequately. The later can be implemented in broader range of 
applications and comes to fill the gap when linear regression is incapable of 
describing complex systems. Additionally, the theory associated with both 
models is well-developed and capable of producing interpretable predictions and 


optimizations. 


The step-wise method embedded in JMP was the main variable 
selection strategy because it creates models with good fit to the data using as 
few predictors as possible. Furthermore, the alpha level, which tests the 
importance of each one of the predictors included in the model in the presence of 
the rest, was set to .01 to avoid over-fitting. This procedure is useful in creating 
parsimonious models (Keymal, 2013). 


Finally, the models are compared in terms of R® and the number of 
factors included in the model. R® is the proportion of the variability of the 
response explained by the regression model. In the output that will be presented 
later, the Adjusted R’ is also shown. This metric attempts to adjust for inflation in 
R? when the number of observations is small compared to the number of factors. 
Therefore, in discussing the results, we focus on adjusted R*. 
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(1) Main Effects Model. First, we fitted a model only with 
main effects using the step-wise method and alpha level of 0.01. Prior to 
examining the results, we first checked that the assumptions of the model were 
met. Observing the Residual by Predicted plot, we notice that the errors are not 
equally scattered across the range of the response variable, which means that 
the variance is not constant (heteroscedasticity). Although the Residual by Raw 
plot indicates that the errors are independent of each other, we concluded that 
the above model did not meet the criteria for hypotheses testing. Both the plots 
are shown in Figure 40. 
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Figure 40. Model diagnostics for the main effects model prior the Box-Cox 
transformation (“Open Sea’). 


For that reason we transformed the input data according to 
Box-Cox result. Figure 41 suggests that we should take the square root of the 
data instead of the original ones. 
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Figure 41. Box-Cox graph that suggests transformation with A=0.5 (“Open Sea’). 


Indeed, after the transformation all the model assumptions 
were met. The Residual by Predicted plot shows constant variance and the 
Residual by Raw plot along with the large Durbin-Watson p-value (0.771) prove 
the independence of the observations. Finally, the Normal Quantile plot shows 
the Normality of the errors (Figure 42). 
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Figure 42. Model diagnostics for the main effects model after the Box-Cox 
transformation (“Open Sea’). 
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The model we ended up with involves 10 factors and gives 
Adjusted R® value of 0.781 (Table 16). The small p-value (<0.001) indicates that 
at least one of the predictors included in the model has great impact on the 


response. 


Table 16. Statistical results for the main effects model after the Box-Cox 
transformation (“Open Sea”). 


4 \~ Response Mean(PercentRedTracked),A(0.5) 


4 Summary of Fit 4 Analysis of Variance 
RSquare 0.798401 Sum of 
RSquare Adj 0.781317 Source OF Squares Mean Square’ F Ratio 
Root Mean Square Error 0.051656 Model 10 1.2469639 0.124696 46.7322 
Mean of Response 0.454564 Error 118 0.3148619 0.002668 Prob>F 
Observations (or Sum Wots) 129 C. Total 128 1.5618257 <.0001" 


The factors of the model were sorted in order of significance 
from high to low (Table 17). The detection ranges of the Blue Sensors seem to 
be the most significant, followed by the Communication Accuracy of the 
Operational Centre and the Average Time between Detections of the Blue 
Radars. The Number and the Speed of Merchant Vessels also have impact to 
the MOE, along with the Stealth of Red Ships, the Average Time between 
Detections of the UAV and the Average Time between Detections of the Blue 
Ships. To give the direction of the impact of each predictor, as the Detection 
Ranges, the Communication Accuracy and the Speed of the Merchant Vessels 
increase, the more effective the surveillance system becomes. In contrast, as the 
Average Time between Detections, the number of Merchant Vessels and the Red 
Stealth decreases, the surveillance system becomes less effective. 
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Table 17. Sorted parameter estimates for the main effects model after the 


Box-Cox transformation (“Open Sea’). 


4 Sorted Parameter Estimates 


Term Estimate 
Radar_MaxDetRange 4.5688e-6 
BlueShips_MaxDetRange 8.9862e-6 
UAV_MaxDetRange 5.5045e-6 
OpsCentre_CommsAccuracy 0.0021806 
Radar_AvgTimeBweenDet -0.000563 
NumberMerchVessels -0.002235 
MerchVessels_ Speed 0.0048106 
Red_Stealth -0.000875 
UAV_AvaTimeBweenDet -0.000427 


BlueShips_AvgTimeBweenDet -0.000195 


Std Error t Ratio 


3.908e-7 
7.817e-7 
6.255e-7 
0.000313 
0.000136 
0.0006 
0.001294 
0.00026 
0.000136 
6.739e-5 


11.69 
11.50 

8.80 

6.97 
-4.15 
-3.73 

3.72 
-3.36 
-3.14 
-2.90 


Prob>|t| 
<.0001* 
=.0001* 
=.0001* 
=.0001* 
=.0001* 
0.0003* 
0.0003* 
0.0010* 
0.0021* 
0.0045* 


(2) | Main Effects Model with Interactions. To achieve a 


better model, we added all the two-way interactions to the previous model. Again 


we conducted model diagnostics, and the results are shown in Figure 43. It can 


be easily seen that all the model assumptions are met. The Residual by 


Predicted plot shows constant variance, and the Residual by Raw plot along with 


the large Durbin-Watson p-value (0.166) proves the independence of the 


observations. Finally, the Normal Quantile plot shows the Normality of the errors. 


No further transformation is needed. 
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Figure 43. Model diagnostics for the main effects with interactions model (“Open 
Sea”). 


The new model contains 13 factors and gives Adjusted R*® 
value of 0.823 (Table 18). The small p-value (<0.001) indicates that at least one 
of the predictors included in the model has great impact to the response. 


Table 18. Statistical results for the main effects model with interactions 
(“Open Sea’). 


4 \~ Response Mean(PercentRedTracked),A(0.5) 


4 Summary of Fit 4 Analysis of Variance 
RSquare 0.841691 Sum of 
RSquare Adj 0.823796 Source OF Squares Mean Square _F Ratio 
Root Mean Square Error 0.046368 Model 13 1.3145754 0.101121 47.0331 
Mean of Response 0.454564 Error 115 0.2472503 0.002150 Prob>F 
Observations (or Sum Wots) 129 C. Total 128 15618257 <.0001* 


The factors of the model with interactions were sorted in 
order of significance from high to low (Table 19). Comparing to the linear model, 
the eight most significant factors are exactly the same, with the same direction of 
impact. In addition, this model suggests that the interaction between the Number 
and the Speed of Merchant Vessels, the Blue Ships’ Average Time between 
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Detections, the interaction between the Blue Radar Range and its Time Between 
Detections, as well as the interaction between the Blue Radar’s Detection Range 
with its Average Time between Detections have a statistically significant, but to a 
lesser extent, effect to the response. Finally, the Red Speed affects also the 
MOE. 


Table 19. Sorted parameter estimates for the main effects model with 
interactions (“Open Sea’). 


4 Sorted Parameter Estimates 



























































Term Estimate StdError t Ratio Prob>|t| 
Radar_MaxDetRange 45773e-6 3.508e-7 13.05 | <.0001* 
BlueShips_MaxDetRange 8.9845e-6 7.017e-7 12.80 | <.0001* 
UAV_MaxDetRange 54965e-6 5615e-7 979 <.0001* 
OpsCentre_CommsAccuracy 0.002183 0.000281 7.78 <.0001* 
Radar_AvgTimeBweenDet -~0.000571 0.000122 -468 <.0001* 
MerchVessels_Speed 0.0048516 0.001162 4.18 <.0001* 
NumberMerchVessels -0.002221 0.000538 -4.13 <.0001* 
Red_Stealth -0.000874 0.000234 -3.74 0.0003* 
(NumberMerchVessels-17.0078)*(MerchVessels_Speed-14.0155) 0.0005419 0.000145 3.74 0.0003* 
UAV_AvgTimeBweenDet -0.000421 0.000122 -3.45 0.0008* 
BlueShips_AvgTimeBweenDet -0.000196 6.05e-5 -3.24 0.0016* 
(Radar_MaxDetRange-30000.2)*(Radar_AvgTimeBweenDet-62.5039) -3.373e-8 1.069e-8 -3.16 0.0020* 
Red_Speed -0.001973 0.0007 -282 0.0057* 


(3) | Model with Quadratic Effects. Finally, we fitted a 
polynomial model to check whether or not second order factors affect the MOE. 
Again we conducted model diagnostics, and the results are shown in Figure 44. It 
can be easily seen that all the model assumptions are met. The Residual by 
Predicted plot shows constant variance, and the Residual by Raw plot along with 
the large Durbin-Watson p-value (0.873) proves the independence of the 
observations. Finally, the Normal Quantile plot shows the Normality of the errors. 


No further transformation is needed. 
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Figure 44. Model diagnostics for the second order model (“Open Sea’). 


The Second Order model contains 13 factors and gives an 
Adjusted R® value of 0.822 (Table 20). Compared to the previous model, we 


notice that both of them give similar values of Adjusted R® with the same number 


of terms. The small p-value (<0.001) indicates that at least one of the predictors 


included in the model has great impact on the response. 


Table 20. Statistical results for the second order model (“Open Sea”). 


4 ~ Response Mean(PercentRedTracked),A(0.5) 


4 Summary of Fit 
RSquare 0.840291 
RSquare Adj 0.822237 
Root Mean Square Error 0.046573 
Mean of Response 0.454564 
Observations (or Sum Wots) 129 


4 Analysis of Variance 
Sum of 
Source DF Squares Mean Square F Ratio 
Model 13 13123883 0.100953 46.5431 
Error 115 0.2494374 0.002169 Prob>F 
C. Total 128 15618257 <.0001* 


The factors of the Second Order model were sorted in order 
of significance from high to low (Table 21). Compared to the previous model, we 


notice that the seven most significant factors are exactly the same. In addition, 


instead of the two interaction terms there is one term with quadratic effect (Blue 
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Radar’s Detection Range). Finally, the Speed of the UAV seems to affect the 
response variable significantly. Thus, all three regression models are consistent 
in terms of which are the most influential predictors. 


Table 21. Sorted parameter estimates for the second order model (“Open 
Sea”). 
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Term Estimate StdError t Ratio Prob>|t| 
Radar_MaxDetRange 45709e-6 3.524e-7 12.97 <.0001* 
BlueShips_MaxDetRange 8.9803e-6 7.048e-7 1274 <.0001* 
UAV_MaxDetRange 5.4885e-6  5.64e-7 9.73 <.0001* 
OpsCentre_Comms<Accuracy 0.0021807 0.000282 7.74 <.0001* 
Radar_AvgTimeBweenDet -0.000569 0.000123 -464 — <.0001* 
MerchVessels_Speed 0.0048326 0.001167 414 <.0001* 
NumberMerchVessels -0.002233 0.000541 -4.13 | <.0001* 
(Radar_MaxDetRange-30000.2)*(Radar_MaxDetRange-30000.2) -1.31e-10 3.39e-11 -3.88 | 0.0002* 
Red_Stealth -0.000872 0.000235 -3.71 0.0003* 
UAV_AvgTimeBweenDet -0.000423 0.000123 -3.45 | 0.0008* 
BlueShips_AvgTimeBweenDet -0.000196 6.076e-5 -3.23 0.0016* 
Red_Speed -0.001975 0.000703 -2.81 0.0058* 
UAV_Speed 0.0004722 0.000176 2.68 0.0085* 


b. Partition Tree 


Next, partition trees were created to more easily understand the 
regression results above. Partition Trees can be easily built using JMP. They are 
flexible because the user can go back and forth by “splitting” or “pruning” them. 
They are also able to handle both numeric and categorical data. Their main 
characteristic is their ability at fitting jumps or plateaus in the data, which 


provides the analyst with an easy-to-understand output. 


To construct the Partition Tree we used the mean values of the 
MOE1 as the response variable and all the factors as input. The first seven splits 
occur on factors that were found to be the most important during the Regression 
Analysis, such as the Detection Ranges of all the Blue Sensors, as well as the 
Radar’s Average Time between Detections. The exception to this is the speed of 
the Red Ships, which emerges as significant. The Partition Tree after the first 
seven splits (R* 0.667) is shown in Figure 45. We observe that the surveillance 
system performs better (Mean=0.43, SD=0.054) when the maximum detection 


ranges of the Blue Radar, ships and UAV are more than 22,188, 15,156 and 
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19,258 meters, respectively, and the speed of the Red Ships is less than 21 kn. 
In contrast, the surveillance system performs worse (Mean=0.077, SD=0.03) 
when the corresponding ranges are less than 22,188, 16,406 and 15,938 meters. 


‘All Rows 
Count 129 LogWorth Difference 
Mean 0.187354 94141206 0.10665 
StdDev 0.1012394 
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Count 23 LogWorth Difference Count 16 Count 45 LogWorth Difference Count 45 LogWorth Difference 
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Figure 45. Partition tree for the “Open Sea” model after seven splits 


Following that we created a Partition Tree with 21 splits, which 
gave R® 0.856. The values of R’ with respect to the number of splits are shown in 
Figure 46. 


4 Split History 


1.00 
0.75 
@ 
o 
Ss 050 
a 
oe 
0.25 
0.00 
0 5 10 15 20 


Number of Splits 


Figure 46. Split history for the Partition tree of the “Open Sea” model. 
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After sorting the variables that contribute to the response variable, 
we get the result of Table 22. Eight out of the ten significant factors that were 
found after fitting the model with the main effects are also indicated by the 
Partition Tree. It is worth noticing that just the Detection Ranges of the Blue 
Sensors explain about 78 percent of the MOE variability. 


Table 22. Variable contribution for the Partition tree of the “Open Sea” model. 


4 Column Contributions 


Number 
Term of Splits ss 
Radar_MaxDetRange 2 0.32560017 
BlueShips_MaxDetRange 2 0.30547547 
UAV_MaxDetRange 4 0.22704193 
Radar_AvgTimeBweenDet 3 0.08647983 
Red_Speed 1 0.05046437 
BlueShips_Speed 1 0.03202279 
Red_Stealth 1 0.02894743 
UAV_AvgTimeBweenDet 1 0.02296974 
UAV_ProbClass 2 0.01281549 
BlueShips_PrClass 1 0.00899291 
NumberMerchVessels 1 0.00993990 
MerchVessels_Speed 1 0.00978825 
OpCentre_CommsLatency 1 0.00230404 


2. “Many Islands” Model 


The “Many Islands” model with one UAV (and specifically the northern 
one), which was created in MANA, used the appropriate DOE that was presented 
in Chapter V. The 129 DP were replicated 500 times, each resulting in 65,000 
observations. Instead of using the raw data, we use the mean values of the 
replicated DP to reduce the calculations and produce more visually efficient 
displays. 
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a. Regression Analysis 


(1) Main Effects Model. First we fitted a model with only 
the main effects of all the factors. Using the stepwise strategy and setting an 
alpha level of 0.01, we came to a model with only eight terms. We conducted 
model diagnostics and we noticed that all the model assumptions are met. The 
Residual by Predicted plot shows constant variance, and the Residual by Raw 
plot along with the large Durbin-Watson p-value (0.589) proves the 
independence of the observations. Finally, the Normal Quantile plot shows the 
Normality of the errors (Figure 47). 


4 Residual by Predicted Plot 4 ~ Durbin-Watson 

Durbin- Number 

Watson of Obs. AutoCorrelation Prob<DW 
2.0306976 129 -0.0167 0.5898 
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Residual 
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Figure 47. Model diagnostics for the main effects model (“Many Islands’). 


The model we ended up with involves eight factors and gives 
an Adjusted R® value of 0.742 (Table 23). The small p-value (<0.001) indicates 
that at least one of the predictors included in the model has great impact on the 


response. 
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Table 23. Statistical results for the main effects model (“Many Islands”). 


§ ~/ Response Mean(PercentRed Tracked) 
4 Summary of Fit 


RSquare 0.758305 
RSquare Adj 0.742192 
Root Mean Square Error 0.049536 
Mean of Response 0.359977 
Observations (or Sum Wgts) 129 


4 Analysis of Variance 
Sum of 
Source OF Squares Mean Square F Ratio 
Model 8 0.9238422 0.115480 47.0616 
Error 120 0.2944571 0.002454 Prob>F 
C. Total 128 1.2182993 <.0001" 


The factors of the model were sorted in order of significance 


from high to low (Table 24). The Communication Accuracy of the Operational 


Centre emerges as the most significant factor followed by the detection ranges of 


the Blue Sensors. The Stealth of the Red Ships comes next, followed by the 


Average Times between Detections of the three Blue Sensors. To give the 


direction of the impact of each predictor, as the Communication Accuracy and 


the Detection Ranges of the Blue Sensors increase, the more effective the 


surveillance system becomes. In contrast, as the Red Stealth and the Average 


Time between Detections decreases, the surveillance system becomes less 


effective. 
Table 24. Sorted parameter estimates for the main effects model (“Many 
Islands”). 

4A Sorted Parameter Estimates 
Term Estimate StdError t Ratio Prob>|t| 
OpsCentre_CommsaAccuracy 0.0033661 0.0003 11.23 <.0001* 
BlueShips_MaxDetRange 6.9471e-6 7.496e-7 9.27 <.0001* 
UAV_MaxDetectionRange 48265e-6 5.997e-7 8.05 <.0001* 
Radar_MaxDetRange 2.2715e-6 3.748e-7 6.06 <.0001* 
Red_Stealth -0.001253 0.00025 -5.02 <.0001* 
UAV_AvgTimeBweenDet 0.000549 0.00013 -422 <.0001* 
Radar_AvgTimeBweenDet -0.000438 0.00013 -3.36 0.0010* 
BlueShips_AvaTimeBweenDet -0.000175 6462e-5 -2.71 0.0076* 
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(2) Main Effects Model with Interactions. To achieve a 
better model, we added all the two-way interactions to the previous model. Again 
we conducted model diagnostics and the results are shown in Figure 48. It can 
be easily seen that all the model assumptions are met. The Residual by 
Predicted plot shows constant variance, and the Residual by Raw plot along with 
the large Durbin-Watson p-value (0.607) proves the independence of the 
observations. Finally, the Normal Quantile plot shows the Normality of the errors. 
No further transformation is needed. 


4 Residual by Predicted Plot 4 ~ Durbin-Watson 
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Figure 48. Model diagnostics for the main effects model with interactions (“Many 
Islands”). 


The new model contains ten factors and gives Adjusted R* 
value of 0.798 (Table 25). The small p-value (<0.001) indicates that at least one 
of the predictors included in the model has great impact on the response. 
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Table 25. Statistical results for the main effects model with interactions 


(“Many Islands”). 


4 ~ Response Mean(PercentRed Tracked) 


4 Summary of Fit 4 Analysis of Variance 


RSquare 0.814299 Sum of 

RSquare Adj 0.798562 Source OF Squares Mean Square F Ratio 
Root Mean Square Error 0.043787 Model 10 0.9920603 0.099206 51.7431 
Mean of Response 0.359977 Error 118 0.2262390 0.001917 Prob>F 
Observations (or Sum Wots) 129 C. Total 128 1.2182993 <.0001" 


The factors of the model with interactions were sorted in 


order of significance from high to low (Table 26). This model involves all the eight 
factors that the previous model had. In addition, the six most significant factors 
are exactly the same. Furthermore, this model suggests that the interactions 
between the Operational Centre’s Communication Accuracy with UAV’s 


Detection Range and the Stealth of the Red ships are also significant. 


Table 26. 


Sorted parameter estimates for the main effects model with 
interactions (“Many Islands’). 


4 Sorted Parameter Estimates 


Term 


OpsCentre_CommsAccuracy 


Estimate 
0.0033653 


Std Error tRatio 
0.000265 1270 























Prob>|t| 
<,0001* 


BlueShips_MaxDetRange 6.947e-6 6.626e-7 1048 <.0001* 
UAV_MaxDetectionRange 48268e-6 5301e-7 9.11 <.0001* 
Radar_MaxDetRange 2.2708e-6 3.313e-7 685 <.0001* 
Red_Stealth -0.001255 0.000221 -5.69 <.0001* 
UAV_AvgTimeBweenDet -0.000549 0.000115 -477 <,0001* 


(UAV_MaxDetectionRange-17500)*(OpsCentre_CommsAccuracy-75.0078) 1.6023e-7 


(OpsCentre_Comms<Accuracy-75.0078)*(Red_Stealth-60.0155) 
Radar_AvgTimeBweenDet 


BlueShips_AvgTimeBweenDet 


-0.000059 
-0.000438 
-0.000176 


3.499e-8 458 
1.533e-5 -3.85 
0.000115 -3.80 
5.712e-5 -3.07 








<.0001* 
0.0002* 
0.0002* 
0.0026* 


(3) | Model with Quadratic Effects. Finally, we fitted a 
polynomial model to check whether or not second order factors affect the MOE. 
Again, we conducted model diagnostics, and the results are shown in Figure 49. 
It can be easily seen that all the model assumptions are met. The Residual by 
Predicted plot shows constant variance, and the Residual by Raw plot along with 
the large Durbin-Watson p-value (0.357) proves the independence of the 
observations. Finally, the Normal Quantile plot shows the Normality of the errors. 


No further transformation is needed. 
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4 Residual by Predicted Plot 4 =~ Durbin-Watson 
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Figure 49. Model diagnostics for second order model (“Many Islands”). 


The last model contains 12 factors and gives Adjusted R® 
value of 0.826 (Table 27). The small p-value (<0.001) indicates that at least one 
of the predictors included in the model has great impact on the response. 


Table 27. Statistical results for the second order model (“Many Islands”). 


4 =| Response Mean(PercentRed Tracked) 


4 Summary of Fit 4 Analysis of Variance 
RSquare 0.842367 Sum of 
RSquare Adj 0.82606 Source DF Squares Mean Square __ F Ratio 
Root Mean Square Error 0.040689 Model 12 1.0262547 0.085521 51.6571 
Mean of Response 0.359977 Error 116 0.1920446 0.001656 Prob>F 
Observations (or Sum Wgts) 129 C. Total 128 12182993 <.0001* 


The factors of the model with interactions were sorted in 
order of significance from high to low (Table 28). Comparing to the previous 
models, we notice that this contains all the main effects we observed previously, 
but instead of the interaction terms, it involves the quadratic effects of the 
Communication Accuracy of the Operational Centre, the Detection Range of the 
Blue Radar and the UAV, as well as the Time between detections of the UAV. 
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Table 28. Sorted parameter estimates for the second order model (“Many 






































Islands”). 
4 Sorted Parameter Estimates 
Term Estimate Std Error tRatio Probe|t| 
OpsCentre_CommsdAccuracy 0.003368 0.000246 13.68 <,0001* 
BlueShips_MaxDetRange 6.9497e-6 6.157e-7 11.29 <.0001* 
UAV_MaxDetectionRange 4823e-6 4.926e-7 9.79 <,0001* 
Radar_MaxDetRange 2.273e-6 3.079e-7 7.38 <.0001* 
Red_Stealth -0.001253 0.000205 -6.11 <.0001* 
UAV_AvaTimeBweenDet -0.000549 0.000107 -5.13 <.0001* 
Radar_AvgTimeBweenDet -0.000438 0.000107 -4.09 <.0001* 
(OpsCentre_CommsAccuracy-75.0078)*(OpsCentre_CommsAccuracy-75.0078)  8.042e-5 0.00002 4.00 0.0001* 
(Radar_MaxDetRange-30000.2)*(Radar_MaxDetRange-30000.2) -1.12e-10 3.18e-11  -3.62 0.0006* 
BlueShips_AvaTimeBweenDet -0.000175 0.000053 -3.30 0.0013* 
(UAV_AvaTimeBweenDet-62.5039)*(UAV_AvaTimeBweenDet-62.5039) 1.205e-5 3.661e-6 329 0.0013* 
(UAV_MaxDetectionRange-17500)*(UAV_MaxDetectionRange-17500) -2.03e-10 7.89e-11  -2.58 0.0112* 


b. Partition Tree 


To construct the Partition Tree, we used the mean values of the 
MOE1, as the response variable, and all the factors as input. The first six splits 
occur on factors that found to be the most important during the Regression 
Analysis, such as the Communication Accuracy of the Operational Centre and 
Detection Ranges of all the Blue Sensors. The Partition Tree after the first six 
splits (R= 0.52) is shown in Figure 50. We observe that the surveillance system 
performs better (Mean=0.49, SD=0.07) when the Communication Accuracy of the 
Operational Centre is greater than 78 percent and the Blue Ships’ Detection 
Range is more than 19,844 meters. In contrast, the surveillance system performs 
worse (Mean=0.25, SD=0.05) when the corresponding values are less than78 
percent and 11,904 meters, respectively. 
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Figure 50. Partition tree for the “Many Islands” model after six splits. 


Following that we created a Partition Tree with 20 splits which gave 


R® 0.821. The values of R® with respect to the number of splits are shown in 


Figure 51. 


4 Split History 


1.00 
0.75 
@ 
o 
Ss 0.50 
= 
oe 
0.25 
0.00 
0 5 10 15 20 


Number of Splits 


Figure 51. Split history for the Partition tree of the “Many Islands” model. 


After sorting the variables that contribute to the response variable, 
we get the result of Table 29. It is worth noticing that the six most significant 
terms that the Partition Tree indicates explain about 94 percent of the MOE 


variability and were also predictors in the Regression Analysis. 
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Table 29. Variable contribution for the Partition tree of the “Many Islands” 
model. 


0.00881418 
0.00864855 
0.00140383 


UAV_CommsLatency 
UAV_ProbClass 
Radar_ProbClass 


4 Column Contributions 

Number 
Term of Splits ss 
OpsCentre_CommsAccuracy 4 0.35727677 
BlueShips_MaxDetRange 4 0.31591995 
UAV_MaxDetectionRange 4 0.12535109 
UAV_AvgTimeBweenDet 1 0.06931791 
Radar_MaxDetRange 1 0.04084012 
Red_Stealth 1 0.02735960 
OpCentre_CommsLatency 1 0.02431740 
BlueShips_PrClass 1 0.02071166 
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Vi. CONCLUSION 


A. DISCUSSION AND OPERATIONAL INSIGHTS 
1. Answers to the Primary Research Question 


The hypothesis we tested was whether “a surveillance system with UAVs 
performs better than one without UAVs assuming that the target’s tactic is to 
approach transiting merchant vessels or to take advantage of nearby shorelines.” 
This hypothesis was confirmed. The simulation we created shows that 
integrating one or two UAVs into a traditional surveillance system makes it more 
efficient in the detection and persistent surveillance of enemies and neutral 
targets. This conclusion applies to all the areas of the Aegean Sea, either the 
Northern and Southern Aegean Sea (modeled by the “Open Sea” model), or the 


Central and Southeast (modeled by the “Many Islands” model). 


In all the cases we studied, we found that a traditional system comprised 
only of surface ships and ground radars performs worse than a system that uses 
UAVs. Additionally, we explored a “futuristic” scenario where only two UAVs 
support the ground radars, without the use of surface ships (no Blue Ships 
scenario). The results showed that we can obtain similar results with this system 
as with the system in which we deploy one UAV and two surface ships in support 
of the ground radars (1-UAV scenario). This conclusion suggests that the 
decision makers should review the surveillance policy. They can shift to less 
reliance on the cost-ineffective use of surface ships towards the increased use of 
UAVs to better monitor maritime areas. This solution might be more feasible in 
adverse weather conditions and when there is lack of operational surface units 


available. 


This study also demonstrated that the potential tactics the enemy might 
use to avoid detection cannot mislead UAVs. Regardless of whether the enemies 
try to take advantage of the nearby merchant vessels and sail close to them to 


transit long distances, or they use the shorelines of numerous islands as 
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coverage, a surveillance system that deploys UAVs has significantly better 
performance than a system that uses only ground radars and surface ships. 


This thesis also shows that the area from where a UAV is launched affects 
the performance of the whole surveillance system. As it is easily understood, the 
more contacts there are close to the launch site, the more the UAV contributes to 
the surveillance system. That is, if the launch site is close to areas with high 
traffic, the UAV will have more opportunities to locate the targets. Additionally, 
the targets will be identified earlier. Therefore, it is extremely important to define 
the best launch site for the UAVs. From the literature review, we found that UAVs 
can be launched either from ground-based sites or appropriately equipped 


surface ships. Both solutions must be examined for strengths and vulnerabilities. 


Furthermore, this study showed that the UAVs can mitigate a vulnerability 
that the ground radar systems inherently have: the incapability to cover their 
“blind sectors” that result from the way systems must be positioned. Because 
monitoring the whole range of the AOI with ground radars is not practical (it 
would require twice as many sensors), the existing radars are placed in areas 
with the most traffic. As a consequence, these systems are not always stationed 
at the highest point of an island, which results in a limited field of view. 
Additionally, taking into account that radar technology is subject to technical 
restrictions, the detecting capabilities are further narrowed. UAVs can easily 


cover those areas, providing information about existing targets. 


Finally, it is worth noticing that the MOE in this study was the percentage 
of time the targets are positively identified. If another MOE had been set, i.e., the 
time until the first detection or classification, then the value of the use of UAVs is 
expected to be further recognized. Their speed and their potential larger 
detection ranges make them the most valuable assets compared to all the other 


traditional surveillance units. 
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2. Answers to the Exploratory Research Question 


This thesis found the important factors that contribute to the maritime 
surveillance system. The two models, which simulated corresponding 
environments in terms of transiting merchant vessels, the existence of nearby 
islands and an enemy’s tactics, were studied separately, but most of the input 
variables appear in both of them. These factors are described below. 


° The detection range of all the sensors that are part of the 
surveillance system seems to have the most significant positive 
effect on the percentage of time the targets are classified. The 
partition tree for the “Open Sea” model highlights the importance of 
having the radars, the UAV and the ships detection ranges larger 
than 22, 19 and 16 km, respectively. 


e Communication Accuracy of the Operational Centre has significant 
positive effect on the system’s performance. The partition tree for 
the “Many Islands” model suggests if this factor has a value of at 
least 78 percent, it will have an ability to better monitor the area of 
interest. It is worth noticing that the Operational Centre is crucial in 
the communication network that we established because its role is 
to gather the information from each one of the sensors and send 
them to the patrolling units. 


° Enemy’s stealth is a factor that has significant negative impact on 
the percentage of time it is being classified. Enemy’s stealth is 
presented in both the scenarios we studied where a target used a 
specific tactic to avoid being detected by the surveillance system. In 
reality, this factor describes the enemy’s attempt to avoid the visual, 
IR and radar’s contact using various methods. 


° The number of merchant vessels and their speed has a negative 
effect on the surveillance system’s performance only in the case in 
which the enemy tries to use them as a “Trojan Horse” to transit 
long distances. In environments where there is high traffic, either 
from merchant or fishing vessels, but the enemy does not try to 
take advantage of it, it seems that the surveillance system is not 
affected. 


° Average Times between Detections of our own sensors are some 
more important factors in terms of the surveillance system’s 
detection capability. All of them positively affect the percentage of 
time the targets are classified regardless the operational 
environment the system operates in. 


° Enemy’s speed is a factor that has a significant negative effect on 
the percentage of time the targets are classified. This factor 
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emerged only in the “Open Sea” model where the target has to 
transit long distances being “exposed” to adversary’s surveillance 
system. This is operationally reasonable because when a target 
sails very close to the shoreline to avoid the detection, it cannot 
maintain a high speed due to navigation dangers. 


UAV speed has a positive effect on the system’s performance only 
in the case where it has to cover large areas. This study showed 
that when the UAV had to cover areas with large number of islands, 
its speed was not important. 


B. FUTURE RESEARCH 


This study was focused on the effectiveness of UAVs when integrated in a 


traditional surveillance system that monitors a complex maritime environment. By 


expanding the field of research or focusing on other key points, the following 


topics might be explored by future studies: 


The requirements for a surveillance system change significantly 
based on the situation in which it operates. Do we need it during 
peace time or war? How do things change if the UAV is capable of 
strike operations and/or it is subject to being hit by enemy targets? 
How vulnerable are the ground radars to attacks? If they are hit, 
how much of the whole system’s performance is affected? A more 
war-focused scenario is believed to be able to provide insights to 
the aforementioned questions. 


This study explored two potential tactics that enemies could have 
used in a particular environment. They aimed to approach either 
large transiting merchant vessels or shorelines of the nearby 
islands. A combination of those, along with other enemy tactics, 
might be further explored. 


Given the complexity of the Aegean Sea and the number/type of 
assets assigned, a study could indicate the optimal placement of 
the ground radars. Additionally, should UAVs be launched from 
particular ground-based sites or from command ships that can be 
deployed wherever the demands are? 


A more detailed study could be done on the cost implications of the 
sensors that participate in the surveillance system. Is it cost- 
effective to use UAVs or surface ships? How many 
UAVs/ships/ground radars should be used? The financial problems 
and the budget constraints that most navies have to face make 
these questions more challenging. 
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C. TAKEAWAY 


This thesis demonstrates that by integrating one or two UAVs into a 
traditional surveillance system, the system becomes more efficient in constantly 
detecting enemies or neutral targets regardless of the complexity of the maritime 
environment in which it operates. Additionally, the potential tactics the enemy 
surface ships might use to avoid detection cannot mislead UAVs. This study 
showed also that the area from where a UAV is launched affects the 
performance of the whole surveillance system; in particular, UAVs can mitigate 
the surveillance system’s inability to cover its “blind sectors,” which are inherent 
in ground radar systems. Finally, the most important factors that affect the 
surveillance system’s performance in all the areas of Aegean Sea are the 
detection range of all the sensors, the communication accuracy of the 
Operational Centre, an enemy’s stealth and the average times between 
detections of our own sensors. Moreover, in areas where the enemy has to 
transit long distances without the existence of many islands nearby, the number 
of merchant vessels and their speed, along with the enemy’s speed and UAV 
speed, emerge as factors that have a significant effect on the percentage of time 
the targets are classified. 
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APPENDIX A. COMPARISON BETWEEN NORTHERN AND 
SOUTHERN UAV FOR BOTH OPEN SEA AND MANY ISLANDS 
MODELS 


In the following sections we provide a detailed comparison between the 
two different options when only one UAV is deployed. In the first scenario, the 
UAV is launched from a location in the Northwest, whereas in the second, the 
UAV started its deployment from the Southeast (“Open Sea” model) or the 
Southwest (“Many Islands” model), respectively. The information that follows 
involves boxplots, ANOVA and tTest results. First, information for the “Open 
Sea” model is presented. In both the cases, statistical analysis shows that the 


location from which the UAV is launched does matter. 
a. ‘Open Sea’ model. 
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t Test 
OpenSea-1UAV(South)-OpenSea-1UAV (North) 
Assuming equal variances 


Difference -0.03376 t Ratio -9.38131 
Std Err Dif 0.00360 DF 998 
Upper CL Dif -0.02670 Prob > |t| <.0001* 
Lower CL Dif -0.04082 Prob >t 1.0000 
Confidence 0.95 Prob <t <.0001* 
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-0.04 -0.02 0.000.01 0.03 





Analysis of Variance 


Source DF Sum of Squares Mean Square F Ratio Prob > F 
Case 1 0.2849663 0.284966 88.0090 <.0001* 
Error 998 3.2314459 0.003238 

C. Total 999 3.5164122 


Means for Oneway Anova 


Level Number Mean Sid Error Lower 95% Upper 95% 
OpenSea-1UAV (North) 500 0.603269 0.00254 0.59828 0.60826 
OpenSea-1UAV(South) 500 0.569507 0.00254 0.56451 0.57450 


Std Error uses a pooled estimate of error variance 


b. ‘Many Islands” model 
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t Test 
Manylslands-1UAV(South)-Manylslands-1UAV (North) 
Assuming equal variances 


Difference -0.01981 t Ratio -5.24457 
Std Err Dif 0.00378 DF 998 
Upper CL Dif -0.01240 Prob > |t| <.0001* 
Lower CL Dif -0.02723 Prob >t 1.0000 
Confidence 0.95 Prob <t <.0001* 
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-0.02 -0.01 0.00 0.01 





Analysis of Variance 


Source DF Sum of Squares Mean Square 
Case 1 0.0981541 0.098154 
Error 998 3.5613916 0.003569 
C. Total 999 3.6595457 

Means for Oneway Anova 

Level Number Mean Sid Error 
Manylslands-1UAV (North) 500 0.722677 0.00267 
Manylslands-1UAV(South) 500 0.702862 0.00267 


Std Error uses a pooled estimate of error variance. 
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F Ratio 
27.5055 


Lower 95% 
0.71743 
0.69762 


Prob > F 
<.0001* 


Upper 95% 
0.72792 
0.70810 
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APPENDIX B. DIAGNOSTICS FOR PARAMETRIC ASSUMPTIONS 
OF MOE 2 IN OPEN SEA MODEL 


Our decision whether to use a parametric or non-parametric method for 
the analysis is based on whether the assumptions of independence, equality of 
variance, and normality are met. Below we study those assumptions for the 
“Open Sea” model with respect to MOE 2. The boxplots and the Histogram of 
Residuals below show that the Equal Variance Assumption and the Near 


Normality Assumption, respectively, hold. 
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APPENDIX C. DIAGNOSTICS FOR PARAMETRIC ASSUMPTIONS 
OF BOTH MOES IN MANY ISLANDS MODEL 


In this appendix we provide the results of our study on whether the 
assumptions of independence, equality of variance, and normality are met. The 
results refer to both MOEs of the “Many Islands” model. First, the results about 
MOE1 are presented. The boxplots and the Histogram of Residuals that follow 
show that the Equal Variance Assumption and the Near Normality Assumption, 


respectively, hold. 
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b. MOE 2 


Residual %MerchantClassified 2 vs. Case 
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